Senior Databricks Data Engineer

• Experience (In Years) 6-9
• Job Location Dublin, Ireland

Job Description

Location: Dublin, Republic of Ireland (Flexible hybrid working)

Employment Type: Permanent or Contract

Summary of the role

Hands on Databricks Data Engineer to support a high impact data transformation initiative focused on automating reporting and dashboard processes.

Responsibilities

Design, build, and maintain scalable data ingestion and transformation pipelines within a Databricks medallion architecture.

• Data Pipeline Development:
  Design, build, and maintain data ingestion and transformation pipelines using Databricks across structured and unstructured data sources.
• Medallion Architecture Implementation:
  Develop and support data pipelines across Bronze, Silver and Gold layers, ensuring data is cleansed, transformed and optimised for downstream reporting and analytics.
• Data Integration & Processing:
  Implement batch and streaming data pipelines using Databricks, Spark, and Azure Data Factory (ADF) to enable efficient data movement and processing.
• Collaboration with Architects & Stakeholders:
  Work closely with Data Architects and business stakeholders to understand requirements and translate them into reliable, maintainable data engineering solutions.
• Data Quality & Validation:
  Support the implementation of data quality checks, validation rules, and monitoring processes to ensure accuracy and reliability of data pipelines.
• Performance Optimisation:
  Assist in optimising data pipeline performance, scalability and cost efficiency within Databricks and cloud environments.
• Data Governance & Standards:
  Follow established best practices for data governance, lineage, and metadata management, ensuring compliance with organisational standards.
• Testing, Deployment & Release Support:
  Support testing, deployment, and release processes, ensuring solutions are robust and production ready.
• Documentation:
  Create and maintain technical documentation, including pipeline logic, data flows, and transformation rules.
• Continuous Improvement:
  Stay up to date with Databricks features and modern data engineering practices, contributing to ongoing improvements in tooling, frameworks, and ways of working.

Qualifications and essential experience

• Education & Experience:
  Bachelor's degree in IT, Computer Science, or a related field, with 3-6 years' experience in data engineering, data warehousing, or related roles.
• Databricks & Data Engineering Skills:
  Hands on experience with Databricks and Spark based data processing, including building and maintaining data pipelines.
• Data Integration & Processing:
  Experience working with ETL/ELT pipelines using Databricks, Azure Data Factory (ADF), or similar frameworks.
• Data Platform & Database Knowledge:
  Experience working with enterprise data stores such as SQL Server, PostgreSQL, Oracle, Synapse, or similar.
• Cloud Experience:
  Exposure to cloud based data platforms (Azure preferred) and distributed data processing environments.
• Business Intelligence Awareness:
  Basic understanding of reporting tools such as Power BI or Tableau, and how data pipelines support analytics and dashboards.
• Industry Experience (Desirable):
  Experience within Financial Services or regulated environments is beneficial.

Skills and attributes

• Hands on Delivery Focus:
  Strong focus on building and delivering data solutions, with the ability to work across multiple pipeline components.
• Problem Solving:
  Ability to troubleshoot issues in data pipelines and processing workflows, identifying root causes and implementing fixes.
• Collaboration & Teamwork:
  Comfortable working within cross functional teams, including architects, engineers, and business stakeholders.
• Attention to Detail:
  Strong attention to detail when working with data quality, transformations, and validation logic.
• Learning Mindset:
  Eagerness to develop skills in Databricks, cloud data platforms, and modern data engineering practices.
• Communication Skills:
  Ability to clearly communicate technical concepts and updates to both technical and non technical stakeholders.

Benefits

• A competitive salary with a generous bonus, private healthcare, life assurance at 4 your annual salary, income protection insurance, and a rewarding pension.
• Professional development opportunities, including online courses and workshops.
• Equal Opportunity Employer - EXL is committed to diversity. Our company does not discriminate based on race, religion, colour, national origin, sex, sexual orientation, age, or disability status.
• Flexible hybrid working model to support work life balance.

To be considered for this role, you must already be eligible to work in the Republic of Ireland.

Leadership Overview

Lead design and optimisation of complex data solutions, oversee data engineering projects, mentor junior engineers, ensure solutions align with organisational goals, and provide strategic guidance.