Databricks Specialist

Data Freelance Hub; Greater London; On-site; GBP 80,000 - GBP 100,000; Full time

Job summary

A data solutions provider in the United Kingdom is seeking a Databricks Specialist for a short-term contract with an immediate start. The role requires strong skills in Databricks, Python, SQL, and DataOps practices, including ETL/ELT frameworks and experience with cloud platforms AWS, Azure, or GCP. You will develop and maintain robust data pipelines, integrate diverse data sources, and collaborate with business teams. Ideal candidates will have significant experience in data engineering and a commitment to data quality and governance.

Qualifications

• Proven experience in building and managing large-scale data pipelines using Databricks.
• Strong programming skills in Python and SQL for data processing.
• Deep understanding of ETL/ELT frameworks and data warehousing.

Responsibilities

• Develop and maintain data pipelines.
• Integrate data from various sources such as APIs and databases.
• Collaborate with teams to understand data requirements.
• Ensure quality and security of ingested data.

Skills

Databricks

Python

SQL

DataOps practices

ETL frameworks

AWS

Azure

GCP

Git

Data Processing

Job description

DataBricks Specialist - Short Term Contract

⭐ - Featured Role | Apply direct with Data Freelance Hub

This role is for a Databricks Specialist on a short-term contract with an immediate start. Key skills include Databricks, Python, SQL, and DataOps practices. Experience with ETL/ELT frameworks and cloud platforms (AWS, Azure, GCP) is required.

United Kingdom

Location: London, England, United Kingdom

#Azure #AWS (Amazon Web Services) #Security #Python #Automated Testing #Delta Lake #Scala #Databricks #Data Processing #Data Engineering #Spark (Apache Spark) #Data Pipeline #Databases #Version Control #GCP (Google Cloud Platform) #PySpark #ETL (Extract #Transform #Load) #Cloud #Programming #GIT #DataOps #SQL (Structured Query Language)

Immediate Start: As a Senior Data Engineer, you will lead the design and development of robust data pipelines, integrating and transforming data from diverse data sources such as APIs, relational databases, and files. Collaborating closely with business and analytics teams, you will ensure high-quality deliverables that meet the strategic needs of our organization. Your expertise will be pivotal in maintaining the quality, reliability, security and governance of the ingested data, therefore driving our mission of Collaboration, Innovation, & Transformation.

Key Responsibilities

• Develop and maintain data pipelines.
• Integrate data from various sources (APIs, relational databases, files, etc.).
• Collaborate with business and analytics teams to understand data requirements.
• Ensure quality, reliability, security and governance of the ingested data.
• Follow modern DataOps practices such as Code Versioning, Data Tests and CI/CD.
• Document processes and best practices in data engineering.

Required Skills and Qualifications

• Proven experience in building and managing large-scale data pipelines in Databricks (PySpark, Delta Lake, SQL).
• Strong programming skills in Python and SQL for data processing and transformation.
• Deep understanding of ETL/ELT frameworks, data warehousing, and distributed data processing.
• Hands‑on experience with modern DataOps practices: version control (Git), CI/CD pipelines, automated testing, infrastructure-as-code.
• Familiarity with cloud platforms (AWS, Azure, or GCP) and related data services.
• Strong problem‑solving skills with the ability to troubleshoot performance, scalability, and reliability issues.
• Proficiency in Git.