Senior Data Engineer

Job Details

Senior Data Engineer
Location: UK Hybrid
Division: Technology

About Verian

Verian is an independent research, evidence, evaluation, and communications agency that helps governments solve public policy challenges by working with citizens and the public realm.

We provide services to government and public sector organizations, drawing on expertise in social research, policy development, and communication.

The agency operates globally, with offices across Europe, Asia Pacific, and the US. 

About the Role

We are seeking a Senior Data Engineer to join our Data Platform team and contribute to the delivery of our product strategy for managing and analysing social research data. 

Working primarily with Azure and Databricks, supported by technologies like Power BI, you’ll design, build, and maintain scalable, reliable, and high-quality data pipelines.

As a key member of the team, you will collaborate with product managers, lead engineers, and cross-functional teams to turn business requirements into technical solutions that drive real-world impact.

Responsibilities

• Collaborate with global business stakeholders to translate business requirements into clear technical specifications
• Design, develop and maintain data pipelines using Azure Data Factory (ADF), Databricks, Synapse, Python, R, Spark, Pandas and SQL
• Manage and optimize Azure resources including Azure Portal, Blob Storage, and Data Lake
• Implement and manage data workflows, ensuring seamless data orchestration and integration
• Develop and maintain ETL processes, including delta loading and change data capture (CDC) mechanisms
• Implement pipelines with Azure DevOps, Terraform and/or CI/CD
• Ensure data security, quality, and governance, including access controls and retention policies
• Use Power BI for data visualisation, reporting, and insights delivery

Skills and Experience

• Proven experience as Senior Data Engineer in international environments, ideally within the consulting sector
• Strong stakeholder management skills and ability to adapt across regions and cultures
• Comfortable working in a fast-paced, agile environment
• Track record in designing and implementing scalable data platforms and products
• Deep expertise in Cloud technologies, especially Microsoft Azure
• Advanced skills in Python, R, Spark, Pandas, and SQL
• Hands-on experience with Delta Live Tables
• Familiarity with the Databricks medallion architecture
• Working knowledge of Unity Catalog and data governance principles
• Experience applying Agile and DevOps practices
• Experience with CI/CD pipelines and Terraform

Why Join Verian?

Joining Verian means becoming part of a dynamic and collaborative community that thrives on innovation and intellectual curiosity.

•    Play a pivotal role in building our data platform from the ground up, at the heart of Verian’s digital transformation.
•    Contribute to innovative data products that leverage AI and Machine Learning.
•    Work on cutting-edge projects that influence public policy and drive meaningful social change.
•    Benefit from flexible working arrangements that support a health work–life balance.
•    Join a supportive, passionate team in an environment where wellbeing truly matters.

How to Apply

If you’re passionate about turning data into insights that impact society, we’d love to hear from you! 

Apply online today, and if your profile matches, we’ll be in touch for an initial exploratory interview.

We encourage applications from all sections of society and provide reasonable accommodations for candidates with disabilities throughout the application and interview process.

Learn more about us at veriangroup.com