Sql jobs in United Kingdom
DataBricks Specialist - Short term contract
Ciandt
DataBricks Specialist - Short term contract
Ciandt
Customer Success & Solution Engineering Senior Solutions Engineer London, UK (Hybrid)
Applied Intuition Inc.
Discover more opportunities than anywhere else. Find more jobs nowSenior Software Engineer
FactSet Research Systems Inc.
Connect with headhunters to apply for similar jobsAnalytics Engineer Team Lead London
Applied Intuition Inc.
Fraud Data Analyst - Fixed-Term Contract (6months)
Data Freelance Hub
Tech Lead - Data Analytics & Data Engineering (AWS) - SC+NPPV3
Data Freelance Hub
DataBricks Specialist - Short term contract
AnaVation LLC
Implementation Lead London, UK
Applied Intuition Inc.
DataBricks Specialist
Job summary
A leading technology company in Greater London is seeking a Senior Data Engineer who will lead the design and development of robust data pipelines. The role includes integrating and transforming data from diverse sources and collaborating closely with analytics teams. Candidates should have proven experience in building large-scale data pipelines, strong programming skills in Python and SQL, and an understanding of modern DataOps practices. Join us in driving innovation and transformation in data management.
Qualifications
- Proven experience in building and managing large-scale data pipelines in Databricks.
- Deep understanding of ETL/ELT frameworks and data warehousing.
- Hands-on experience with version control (Git) and CI/CD pipelines.
Responsibilities
- Develop and maintain data pipelines.
- Integrate data from various sources such as APIs and databases.
- Collaborate with teams to understand data requirements.
Skills
As a Senior Data Engineer, you will lead the design and development of robust data pipelines, integrating and transforming data from diverse data sources such as APIs, relational databases, and files. Collaborating closely with business and analytics teams, you will ensure high-quality deliverables that meet the strategic needs of our organization. Your expertise will be pivotal in maintaining the quality, reliability, security and governance of the ingested data, therefore driving our mission of Collaboration, Innovation, & Transformation.
- Develop and maintain data pipelines.
- Integrate data from various sources (APIs, relational databases, files, etc.).
- Collaborate with business and analytics teams to understand data requirements.
- Ensure quality, reliability, security and governance of the ingested data.
- Follow modern DataOps practices such as Code Versioning, Data Tests and CI/CD.
- Document processes and best practices in data engineering.
- Proven experience in building and managing large-scale data pipelines in Databricks (PySpark, Delta Lake, SQL).
- Strong programming skills in Python and SQL for data processing and transformation.
- Deep understanding of ETL/ELT frameworks, data warehousing, and distributed data processing.
- Hands-on experience with modern DataOps practices: version control (Git), CI/CD pipelines, automated testing, infrastructure-as-code.
- Familiarity with cloud platforms (AWS, Azure, or GCP) and related data services.
- Strong problem-solving skills with the ability to troubleshoot performance, scalability, and reliability issues.
* The salary benchmark is based on the target salaries of market leaders in their relevant sectors. It is intended to serve as a guide to help Premium Members assess open positions and to help in salary negotiations. The salary benchmark is not provided directly by the company, which could be significantly higher or lower.