Senior / Lead Data Engineer opportunity with leading data consultancy based in Dublin 2 (hybrid working 2-3 days). Our client delivers cutting-edge analytics and AI solutions to enterprise-level customers. Responsibilities
Technical project delivery and hands-on execution of complex Databricks and Azure solutions
Develop and performance tune data pipelines using SQL and PySpark.
Implement and manage Databricks Unity Catalog.
Develop reports and reconciliation dashboards to validate data.
Manage data quality frameworks
Manage the implementation of Git and DevOps (CI/CD) standards
Work directly with clients technical stakeholders to manage expectations regarding timelines and technical feasibility.
Lead daily stand-ups and sprint planning for your project team
Experience
5+ years hands-on experience in Data Engineering
2+ years exp working on large-scale Databricks deployments on Azure
Proficiency in PySpark and SQL for complex data transformation and analysis
Proven experience as a lead on projects, sprint planning and managing client communication.
Demonstrable experience leading the technical implementation of Git integration, Databricks, and CI/CD pipelines within an Azure DevOps environment.
Strong communication skills and confidence working directly with clients
Relevant certifications in Azure or Databricks are desirable