Senior Data Engineer: Build Scalable Pipelines

Optima Partners; City of Edinburgh; On-site; GBP 60,000 - 80,000; Full time

Job summary

A data and business consultancy in Edinburgh is seeking a Senior Data Engineer to design and optimize data pipelines and architectures. You will work in cross-functional teams to implement advanced data solutions, driving technical excellence and mentoring junior engineers. Ideal candidates will have over 5 years of experience in data engineering and proficiency in relevant programming languages. This role offers a dynamic environment with opportunities for professional growth.

Qualifications

• Proficiency in programming languages like Python, Java, or Scala.
• Hands-on experience with big data technologies such as Databricks and Apache Spark.
• Familiarity with cloud platforms like AWS, Azure, or GCP.
• Expertise in relational databases like Postgres or SQL Server.
• Experience with data warehousing tools such as Snowflake or Redshift.
• 5+ years of experience in data engineering.

Responsibilities

• Design and implement scalable data pipelines.
• Optimize data architectures including data lakes and warehouses.
• Maintain reliable ETL/ELT workflows.
• Collaborate with data scientists and client stakeholders.
• Monitor and enhance performance of data systems.
• Mentor junior engineers and lead initiatives.

Skills

Proficiency in Python, Java, or Scala

Experience with Databricks or Apache Spark

Familiarity with AWS, Azure, or GCP

Expertise in relational databases

Knowledge of Airflow or Talend

Experience with data warehousing tools

Familiarity with Git and Docker

5+ years in data engineering

Job description

A data and business consultancy in Edinburgh is seeking a Senior Data Engineer to design and optimize data pipelines and architectures. You will work in cross-functional teams to implement advanced data solutions, driving technical excellence and mentoring junior engineers. Ideal candidates will have over 5 years of experience in data engineering and proficiency in relevant programming languages. This role offers a dynamic environment with opportunities for professional growth.