A large public sector organisation operates an enterprise data platform that consolidates policy, claims and operational data. The role exists to provide technical leadership across the data engineering teams, combining hands-on pipeline development in Azure Synapse, Azure Data Factory and Databricks with architectural alignment and standards enforcement for the wider platform in Brussels. You will design and optimize data pipelines within the Azure ecosystem, using PySpark and SQL, and act as the primary technical reference between engineers and architects.
The mission
The organisation is migrating and maturing its enterprise data warehouse and analytics layers on Azure, with ingestion and transformation implemented in Azure Data Factory (including Mapping Data Flow), Azure Synapse and Databricks. The work covers structured insurance and operational datasets, production ETL that supports reporting and BI consumers, and a drive to improve performance, data quality and operational reliability across the platform.
In this Technical Lead role you will split time between hands-on delivery and technical guidance. On a typical day you will review and approve pull requests, implement or optimise PySpark jobs on Databricks, translate platform architecture into concrete engineering standards, and coach multiple data engineering squads. Your work feeds directly into the DWH and semantic layers used by downstream Power BI reports and analytics teams.
Your responsibilities
- Lead and enforce technical standards across data engineering teams, ensuring architectural alignment with the data platform architects
- Deliver and optimise production data pipelines, improving throughput, latency and operational stability using Azure Synapse, ADF and Databricks
- Review code and pull requests, provide actionable feedback, and raise engineering quality through peer review and automated checks
- Translate architectural guidelines into implementable patterns for ingestion, transformation and consumption layers
- Coach and mentor junior and mid-level data engineers, running knowledge-transfer sessions and shaping onboarding material
- Drive performance tuning, data quality initiatives and incident root-cause analysis to reduce repeat operational events
Your profile
Essential skills
- Proven senior experience designing, developing and optimising data pipelines within the Azure ecosystem, including Azure Synapse, Azure Data Factory (ADF) and Mapping Data Flow (MDF)
- Hands-on Databricks and PySpark development experience, plus strong SQL skills for transformation and troubleshooting
- Practical knowledge of dimensional modelling and data warehouse concepts, including star schema design and conformed dimensions
- Demonstrated ability to act as a technical lead: code reviews, architectural translation, and cross-team coordination
- Strong communicator able to present technical decisions to non-technical stakeholders and coach engineering teams
Preferred skills
- Experience with Power BI and semantic data modelling for consumption layers
Languages
- English, CEFR C1 or higher
- French, CEFR C1 or higher
- Dutch, CEFR B2