Leave us your email address and we'll send you all the new jobs according to your preferences.
Databricks Architect
Posted 28 minutes 11 seconds ago by Project Recruit
Our client, a leading global supplier for IT services, requires Databricks Architect to be based at their client's office in London or Manchester, UK.
This is a hybrid role - you can work remotely in the UK and attend the London or Manchester office 3 days per week .
This is a 4+ month temporary contract to start asap
Day rate: Competitive Market rate
You will build, secure, automate, and run enterprise Databricks Lakehouse platforms for banking and financial services clients. The role covers workspace provisioning, infrastructure-as-code, CI/CD, security hardening, and platform operations across multiple environments and regions.
Key Responsibilities- Build, configure, and maintain enterprise Databricks Lakehouse platforms across Dev, Test, UAT, and Production environments.
- Provision and manage Databricks workspaces, compute resources, cluster configurations, and platform-level settings.
- Automate infrastructure provisioning and configuration using Terraform / Infrastructure as Code (IaC).
- Develop and maintain CI/CD pipelines for Databricks platform configurations, notebooks, jobs, and application deployments.
- Configure and administer Unity Catalog, including catalogs, schemas, permissions, and access controls.
- Implement RBAC, service principals, secrets management, and least-privilege access for secure platform operations.
- Integrate Databricks with cloud services such as AWS or Azure, including storage, networking, identity, and security services.
- Configure and maintain cluster policies, job clusters, compute configurations, and environment standards.
- Monitor platform health, workloads, jobs, and compute resources and troubleshoot production issues.
- Implement logging, monitoring, alerting, and operational processes to improve platform reliability.
- Optimize Databricks workloads and compute configurations for performance and cost efficiency.
- Automate repetitive platform administration, deployment, and operational activities.
- Support multi-environment and multi-region deployments, ensuring consistent platform configurations.
- Work with Data Engineering, Cloud, DevOps, Security, Governance, and Application teams on platform integration.
- Maintain technical documentation, deployment standards, operational procedures, and platform configurations.
- Support security hardening and governance requirements appropriate for banking and financial-services environments.
- Databricks & Lakehouse
- Strong hands-on experience with:
- Databricks Notebooks
- PySpark DataFrame API
- SQL transformations
- Delta Lake
- Databricks Jobs, Job Flows, Workflows
- Declarative Pipelines (dpx files, pipeline definitions)
- Experience with Unity Catalog
- Databricks Autoloader and Structured Streaming
- Terraform or IaC experience for Databricks configuration (Optional)
- Experience building both batch and incremental pipelines.
- AWS Stack
- Strong familiarity with:
- S3, IAM, KMS
- Glue / Lambda
- Experience migrating workloads from AWS native tools to Databricks.
- General Data Engineer
- Distributed data processing and performance tuning.
- Strong knowledge of Lakehouse medallion architecture
- Data modeling for Lakehouse: bronze silver gold patterns.
- Experience with Git, CI/CD.
Project Recruit
Related Jobs
Sales Support Representative
- Lancashire, Manchester, United Kingdom, M21 0
Oceanographer
- Cambridgeshire, Cambridge, United Kingdom, CB1 0
Production Operator
- Bedfordshire, United Kingdom
Quality Control Technician (Progression to Supervisor)
- £30,000 - £35,000 Annual
- Lancashire, Manchester, United Kingdom, M21 0
Profound and Multiple Learning Difficulties Teaching Assistant
- West Midlands, Solihull, United Kingdom, B91 1