SC Cleared Databricks Data Engineer

Job Title: SC Cleared Databricks Data Engineer

Location: Client sites, remote, or hybrid, depending on project need.

Contract Type: Contract

About Us

Triad Group Plc is an award-winning digital, data, and solutions consultancy with over 35 years' experience primarily serving the UK public sector and central government. We deliver high-quality solutions that make a real difference to users, citizens and consumers.

At Triad, collaboration thrives, knowledge is shared, and every voice matters. Our close-knit, supportive culture ensures you're valued from day one. Whether working with cutting-edge tech or shaping strategy for national-scale projects, you'll be trusted, challenged, and empowered to grow.

We nurture learning through communities of practice and encourage creativity, autonomy, and innovation. If you're passionate about solving meaningful problems with smart and passionate people, Triad could be the place for you.

• Glassdoor score of 4.7
• 96% of our staff would recommend Triad to a friend
• 100% CEO approval

See for yourself some of the work that makes us all so proud:

• Helping law enforcement with secure intelligence systems that keep the UK safe
• Supporting the UK's national meteorological service in leveraging supercomputers for next-level weather forecasting
• Assisting the British government department that is responsible for the safety of consumer products, with systems to track unsafe products
• Powering systems that help the government monitor and reduce greenhouse gas emissions from commercial transport

Role Summary

We are seeking an experienced SC Cleared Databricks Data Engineer to provide support for a data integration initiative where datasets from multiple sources need to be consolidated and pipelined into our Salesforce solution. The goal is to ensure seamless data flow, accuracy, and scalability within the Salesforce environment.

Key Responsibilities

• Design, develop, and maintain data pipelines using Databricks and Apache Spark.
• Implement ETL/ELT processes for structured and unstructured data.
• Optimise data workflows for performance and scalability.
• Collaborate with data scientists and analysts to enable advanced analytics and ML models.
• Ensure data quality, governance, and security across all platforms.
• Integrate data from multiple sources (cloud, on-prem, APIs) into a unified architecture.
• Monitor and troubleshoot data pipelines to ensure reliability and efficiency.
• Contribute to data architecture design and best practices.

Required Skills & Experience

• Proven experience as a Data Engineer or similar role.
• Strong proficiency in Databricks, Apache Spark, and PySpark.
• Hands-on experience with cloud platforms (Azure, AWS, or GCP).
• Solid understanding of data warehousing concepts and SQL.
• Experience with Delta Lake, Lakehouse architecture, and data modelling.
• Familiarity with CI/CD pipelines and version control (Git).
• Knowledge of data governance, security, and compliance standards.
• Excellent problem-solving and communication skills.

Preferred Qualifications

• Experience with Azure Databricks or AWS Databricks.
• Knowledge of MLflow, Airflow, or other orchestration tools.
• Exposure to machine learning workflows and data science collaboration.
• Relevant certifications (eg, Databricks Certified Data Engineer Associate/Professional).

Other Information

If this role is of interest to you or you would like further information, please contact Ben Fowler or submit your application now!

Triad is an equal opportunities employer and welcomes applications from all suitably qualified people regardless of sex, race, disability, age, sexual orientation, gender reassignment, religion, or belief. Triad Group Plc acts as an Employment Business for this contract position.