Senior Data Engineer — Azure Databricks Expert

Applied Intuition Inc.; Greater London; On-site; GBP 70,000 - 90,000; Full time

Job summary

A global investment firm is seeking a Senior Data Engineer to build scalable data pipelines using Azure technologies, particularly Databricks. The successful candidate will have 5+ years of experience in data engineering and must be adept in Python and SQL. Responsibilities include designing data transformation layers, ensuring data quality, and leading best practices in a cloud-native environment. Join a dynamic team to contribute to innovative data solutions!

Qualifications

• 5+ years of experience in data engineering roles with production-grade data platforms.
• Strong hands-on experience with Azure Databricks and performance optimisation.
• Familiarity with developing batch and streaming data processing solutions.

Responsibilities

• Design and build scalable data pipelines on Azure using Databricks.
• Implement data transformation layers for analytic models.
• Ensure data quality and reliability through monitoring and testing.

Skills

Python

PySpark

SQL

Azure Data Factory

Azure DevOps

Data modelling

Data quality

Agile methodologies

Tools

Azure Databricks

CI/CD pipelines

Job description

A global investment firm is seeking a Senior Data Engineer to build scalable data pipelines using Azure technologies, particularly Databricks. The successful candidate will have 5+ years of experience in data engineering and must be adept in Python and SQL. Responsibilities include designing data transformation layers, ensuring data quality, and leading best practices in a cloud-native environment. Join a dynamic team to contribute to innovative data solutions!