Azure Data Engineer

Softcat plc.; Manchester; Hybrid; GBP 60,000 - 80,000; Full time

Job summary

A leading IT infrastructure provider in Manchester is seeking an Azure Data Engineer to design and develop data pipelines and improve data flow for machine learning models. The ideal candidate should have strong experience with Azure tools and Python, and will enjoy flexible working patterns. If you want to be part of a collaborative and supportive team, we encourage you to apply now.

Benefits

Flexible working hours

Hybrid working model

Support for disabilities and neurodiversity

Qualifications

• Strong hands-on experience with Azure Data Factory.
• Strong hands-on experience with Azure Databricks.
• Strong SQL development skills with performance tuning.
• Proficiency in Python for data engineering tasks.
• Proficiency in DBT for data modelling.
• Experience with Azure DevOps for source control.

Responsibilities

• Design, develop, test, and maintain data pipelines.
• Prepare, clean, and transform unstructured and semi-structured data.
• Develop Python-based ETL/ELT scripts and automation tools.
• Implement CI/CD pipelines using Azure DevOps.
• Collaborate with Data Scientists and AI/ML Engineers.

Skills

Azure Data Factory

Azure Databricks

SQL development

Python

DBT

Azure DevOps

Job description

Azure Data Engineer – Softcat (Manchester, UK)

As one of the UK's leading IT infrastructure providers and a FTSE 250 listed company, Softcat is investing heavily in a technology strategy that focuses on cloud, automation, and data‑driven decision making. The Azure Data Engineer will play a key role in shaping how data is transformed and delivered across the organization.

Responsibilities

• Design, develop, test, and maintain robust, reusable data pipelines using Azure Data Factory, Azure Databricks, and DBT.
• Prepare, clean, and transform unstructured and semi‑structured data for LLM training, fine‑tuning, and prompt engineering workflows.
• Develop Python‑based ETL/ELT scripts, data transformation utilities, and automation tools.
• Implement CI/CD pipelines using Azure DevOps and Databricks Asset Bundles to promote automation and reproducibility.
• Collaborate with Data Scientists, AI/ML Engineers, and Analysts to optimise the flow of data into ML and LLM models.

Qualifications

• Strong hands‑on experience with Azure Data Factory (pipelines, triggers, parameterisation, linked services).
• Strong hands‑on experience with Azure Databricks (PySpark, Spark SQL, Delta Lake, performance tuning).
• Strong SQL development skills, including performance tuning and working with large datasets.
• Proficiency with Python for data engineering tasks (Pandas, PySpark, data cleaning, API integrations).
• Proficiency with DBT (data modelling, macros, testing, documentation).
• Experience with Azure DevOps for Git‑based source control and deployment pipelines for data solutions.

Work in a way that works for you

We recognise that everyone is different and offer flexible working patterns:

• Hybrid working
• Flexible hours – adjust the times you start and finish during the day
• Flexibility around school pick‑up and drop‑offs

Working with us

Wherever you work, you will experience freedom and autonomy to realise your potential. You will feel supported by a team that celebrates individuality, encourages different perspectives, and embraces every background.

Join us

To become part of the success story, please apply now.

Disability and neurodiversity support

If you have a disability or neurodiversity, we can provide support or adjustments that you may need throughout our recruitment process or any mitigating circumstance you wish for us to consider. Any information you share on your application will be treated in confidence.