Job Details
Experience: 7-9 Years
Sector:
Data Engineer
Our exclusive Client – a global financial services company based in Dublin city centre – have an opening for a Lead Data Engineer (Databricks). This person will lead and manage the data team, and take ownership for rollout of key Databricks solution on a core product for the business.
This is a hybrid working role (2 days in office / week) in Dublin city centre
Responsibilities
- Lead and manage the Data team (local and dispersed) and collaborate with stakeholders to deliver robust, data-driven outcomes.
- Design of end-to-end data pipelines using Databricks (Spark, Delta Lake, MLflow).
- Integrate AWS services such as S3 / Glue / Lambda
- Ensure compliance with industry standards (e.g., GDPR, SOC, ISO) and internal security policies.
- Optimise Spark performance, automate infrastructure (Terraform/CloudFormation)
- Champion CI/CD, observability, and agile data delivery.
Requirements
- 8+ years relevant experience with 2+ years of Databricks (Spark)
- Strong Python (PySpark) or Scala skills, plus solid SQL expertise.
- Experience is agile data delivery
- Proven leadership in building and managing high-scale, production-grade data environments.
- Certifications in Databricks or AWS are a plus.