Lead Data Engineer

Lead Data Engineer

Location: Edinburgh or Manchester Flexible/Hybrid

Type: Contract

Day Rate: Inside IR35

About the Role

We are seeking a highly experienced Lead Data Engineer to join a growing data team, taking ownership of complex delivery outcomes across enterprise-scale data platforms. This is a hands-on technical leadership role suited to someone who thrives in challenging environments, enjoys solving complex problems, and can independently drive initiatives from concept through to production.

You will be trusted to define technical approaches, coordinate delivery activities, make key engineering decisions, and provide guidance to other Data Engineers while working with modern cloud and data technologies.

Key Responsibilities

• Lead the design, development, and optimisation of large-scale data engineering solutions.
• Take ownership of complex technical challenges and drive delivery with minimal supervision.
• Define delivery approaches, coordinate engineering activities, and ensure successful project outcomes.
• Provide technical leadership and mentoring to Data Engineers across the team.
• Build and maintain scalable data pipelines and data processing solutions.
• Support and optimise Databricks platforms in production environments.
• Drive engineering best practices across CI/CD, testing, automation, and infrastructure management.
• Collaborate with stakeholders to translate business requirements into technical solutions.
• Identify and remove delivery blockers while operating effectively within ambiguous environments.
• Contribute to the evolution of enterprise data platforms and cloud architectures.

Essential Skills & Experience

• Extensive hands-on Experience as a Lead Data Engineer or Senior Data Engineer.
• Expert knowledge of Databricks, including:
  • Platform engineering
  • Data pipeline development
  • Performance optimisation
  • Production support
• Strong Python development skills with Experience building robust, maintainable, and well-tested solutions.
• Proven Experience with Microsoft Azure services and cloud-native data platforms.
• Strong Experience with Terraform and Infrastructure as Code (IaC) practices.
• Excellent knowledge of CI/CD principles, automated deployment pipelines, and release management.
• Experience working within enterprise-scale and regulated environments.
• Ability to work autonomously and make effective technical decisions.
• Strong stakeholder management and communication skills.

What We're Looking For

The ideal candidate will:

• Be a self-sufficient technical leader who requires minimal oversight.
• Be comfortable owning complex problems and delivering outcomes independently.
• Thrive in fast-paced, evolving environments where requirements may be ambiguous.
• Have a passion for engineering excellence and continuous improvement.
• Enjoy mentoring and supporting other engineers while remaining hands-on technically.

Technical Stack

• Databricks
• Python
• Microsoft Azure
• Terraform
• Infrastructure as Code (IaC)
• CI/CD Pipelines
• Enterprise Data Engineering

Why Apply?

This is an opportunity to play a key role in shaping and delivering modern cloud-based data platforms while working on large-scale, business-critical data initiatives. You'll have the autonomy to lead technical solutions, influence engineering standards, and make a significant impact across a highly skilled data engineering function.

Job Title: Lead Data Engineer

Location: Edinburgh, UK

Job Type: Contract

Trading as TEKsystems. Allegis Group Limited, Bracknell, RG12 1RT, United Kingdom. No Allegis Group Limited operates as an Employment Business and Employment Agency as set out in the Conduct of Employment Agencies and Employment Businesses Regulations 2003. TEKsystems is a company within the Allegis Group network of companies (collectively referred to as "Allegis Group"). Aerotek, Aston Carter, EASi, Talentis Solutions, TEKsystems, Stamford Consultants and The Stamford Group are Allegis Group brands. If you apply, your personal data will be processed as described in the Allegis Group Online Privacy Notice available at our website.

To access our Online Privacy Notice, which explains what information we may collect, use, share, and store about you, and describes your rights and choices about this, please go our website.

We are part of a global network of companies and as a result, the personal data you provide will be shared within Allegis Group and transferred and processed outside the UK, Switzerland and European Economic Area subject to the protections described in the Allegis Group Online Privacy Notice. We store personal data in the UK, EEA, Switzerland and the USA. If you would like to exercise your privacy rights, please visit the "Contacting Us" section of our Online Privacy Notice on our website for details on how to contact us. To protect your privacy and security, we may take steps to verify your identity, such as a password and user ID if there is an account associated with your request, or identifying information such as your address or date of birth, before proceeding with your request. commitments under the UK Data Protection Act, EU-U.S. Privacy Shield or the Swiss-U.S. Privacy Shield.