Data Engineer
CV-LibraryWC1A, Bloomsbury Square, Greater LondonpermanentPosted: 30 June 2026
Apply NowESSENTIAL ROLES & RESPONSIBILITIES
* Identify and understand customer data-centric use cases within regulated financial services environments
* Design and implement data ingestion, processing, and transformation pipelines on Azure
* Build and maintain data pipelines for cleaning, normalisation, enrichment, and preparation
* Apply appropriate data modelling techniques and architecture patterns, with a strong focus on medallion architecture
* Orchestrate, monitor, and optimise Azure Databricks jobs and Azure Data Factory pipelines across development, UAT, and production environments
* Operationalise workflows developed by analysts and data scientists
* Support customers in adopting Azure data, analytics, and machine learning services
GOVERNANCE & REPORTING
* Maintain accurate documentation of data pipelines, schemas, transformations, and deployment processes
* Support data governance initiatives including lineage, metadata management, and access control
TECHNOLOGY STACK (AZURE)
Data Engineering & Analytics:
* Azure Databricks (development, UAT, and production)
* Azure Data Factory
* Azure Synapse Analytics (where applicable)
Databases:
* Microsoft SQL Server / Azure SQL Database (primary platforms)
* PostgreSQL (limited use)
* MySQL (limited use)
Security & Governance:
* Role-based access control (RBAC)
* Data encryption and key management
* Audit logging and monitoring
CRITICAL COMPETENCIES – TECHNICAL FIT
Essential:
* Strong SQL skills
* Programming experience with Python and/or Scala
* Hands-on experience with Azure-based data platforms
* Experience designing, building, and maintaining data pipelines
* Strong understanding of data modelling (relational and analytical), including medallion architecture
* Experience orchestrating and optimising Databricks and Data Factory workloads
* Experience using CI/CD pipelines for data and analytics solutions
* Strong awareness of security, networking best practices, GDPR, and PII handling
Desirable:
* Experience with Azure Databricks in production environments
* Familiarity with Azure Machine Learning and AI services
SHIFT & WORKING PATTERN
* Standard business hours, with participation in an on-call rota as required
* Occasional weekend engineering coverage will be required, typically limited to a small number of planned weekends per year to support business continuity, resilience testing, or disaster recovery activities
* Identify and understand customer data-centric use cases within regulated financial services environments
* Design and implement data ingestion, processing, and transformation pipelines on Azure
* Build and maintain data pipelines for cleaning, normalisation, enrichment, and preparation
* Apply appropriate data modelling techniques and architecture patterns, with a strong focus on medallion architecture
* Orchestrate, monitor, and optimise Azure Databricks jobs and Azure Data Factory pipelines across development, UAT, and production environments
* Operationalise workflows developed by analysts and data scientists
* Support customers in adopting Azure data, analytics, and machine learning services
GOVERNANCE & REPORTING
* Maintain accurate documentation of data pipelines, schemas, transformations, and deployment processes
* Support data governance initiatives including lineage, metadata management, and access control
TECHNOLOGY STACK (AZURE)
Data Engineering & Analytics:
* Azure Databricks (development, UAT, and production)
* Azure Data Factory
* Azure Synapse Analytics (where applicable)
Databases:
* Microsoft SQL Server / Azure SQL Database (primary platforms)
* PostgreSQL (limited use)
* MySQL (limited use)
Security & Governance:
* Role-based access control (RBAC)
* Data encryption and key management
* Audit logging and monitoring
CRITICAL COMPETENCIES – TECHNICAL FIT
Essential:
* Strong SQL skills
* Programming experience with Python and/or Scala
* Hands-on experience with Azure-based data platforms
* Experience designing, building, and maintaining data pipelines
* Strong understanding of data modelling (relational and analytical), including medallion architecture
* Experience orchestrating and optimising Databricks and Data Factory workloads
* Experience using CI/CD pipelines for data and analytics solutions
* Strong awareness of security, networking best practices, GDPR, and PII handling
Desirable:
* Experience with Azure Databricks in production environments
* Familiarity with Azure Machine Learning and AI services
SHIFT & WORKING PATTERN
* Standard business hours, with participation in an on-call rota as required
* Occasional weekend engineering coverage will be required, typically limited to a small number of planned weekends per year to support business continuity, resilience testing, or disaster recovery activities