Data Engineering Manager

Alter Domus; Greater London; On-site; GBP 60,000 - 80,000; Full time

Job description

Select how often (in days) to receive an alert: Create Alert

As a world leading provider of integrated solutions for the alternative investment industry, Alter Domus (meaning “The Other House” in Latin) is proud to be home to 90% of the top 30 asset managers in the private markets, and more than 6,000 professionals across 24 jurisdictions.

With a deep understanding of what it takes to succeed in alternatives, we believe in being different - in what we do, in how we work and most importantly in how we enable and develop our people. Invest yourself in the alternative, and join an organization where you progress on merit, where you can speak openly with whoever you are speaking to, and where you will be supported along whichever path you choose to take.

Find out more about life at Alter Domus at careers.alterdomus.com

JOB DESCRIPTION

Join our dynamic and innovative team as a Data Engineering Manager, where you will lead the design, development, and maintenance of our data infrastructure and pipelines. In this pivotal role, you will oversee a team of data engineers, collaborating closely with data analysts, data scientists, and various business stakeholders to ensure that our data is reliable, efficient, and accessible in a scalable manner.

Key Responsibilities

• Lead, mentor, and develop a team of data engineers, fostering a culture of collaboration, innovation, and continuous improvement.
• Set clear goals and performance expectations for team members, conducting regular performance reviews and providing constructive feedback.
• Oversee the design, building, and maintenance of end-to-end data pipelines using Airflow (including managed services like Amazon MWAA) to orchestrate, schedule, and monitor batch/streaming workflows.
• Ensure the implementation of DAGs (Directed Acyclic Graphs) with retry logic, error handling, and alerting to maintain data quality and pipeline reliability.
• Data Ingestion & Transformation
  • Guide the integration of data from various sources using Airbyte for ingestion and dbt for transformations in a scalable and modular fashion.
  • Collaborate with Data Analysts and Data Scientists to implement transformations and business logic, ensuring data is analytics-ready.
• Data Modeling & Warehousing
  • Direct the design and implementation of efficient data models for both structured and semi-structured data in AWS S3 (data lake) and Snowflake (data warehouse).
  • Utilize Databricks for data processing and analytics, enabling real-time insights and advanced analytics capabilities.
  • Ensure data schemas and transformations support advanced analytics, BI reporting, and machine learning use cases.
• Data Governance & Security
  • Lead the utilization of AWS Lake Formation APIs and best practices to maintain data security, access controls, and compliance.
  • Collaborate with IT security to establish robust encryption standards, audit trails, and identity/role-based access.
• Performance Optimization
  • Oversee the optimization of AWS Athena queries and configurations (e.g., data partitioning) for performance and cost efficiency.
  • Monitor and tune Airflow DAGs, Snowflake queries, and Databricks jobs to improve throughput and reliability.
• Collaboration & Stakeholder Management
  • Partner with cross-functional teams, including DevOps, Platform Engineering, and Data Science, to ensure seamless integration of data workflows and systems.
  • Communicate technical solutions effectively to non-technical stakeholders and leadership, translating requirements into actionable tasks.
• Participate in architecture reviews, code reviews, and troubleshooting sessions to ensure quality and alignment with best practices.
• Stay current with emerging trends in data engineering, orchestration tools (Airflow, MWAA), and cloud services (AWS, Snowflake, Databricks).

Qualifications & Skills

• Education & Experience
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • 5+ years of experience in data engineering or a similar role, with at least 2 years in a leadership or managerial capacity working with cloud-based data platforms.
• Technical Skills
  • Cloud & Orchestration:
    • Proficient in Airflow (self-managed or managed services like Amazon MWAA) for workflow orchestration, DAG development, and scheduling.
    • Familiarity with best practices for Airflow DAG structure, dependency management, and error handling.
  • AWS Expertise:
    • Hands‑on experience with AWS Lake Formation, S3, Athena, and related services (e.g., Lambda, Glue, IAM).
  • Snowflake:
    • Proficient in setting up data warehouses, configuring security, and optimizing queries on Snowflake.
  • Databricks:
    • Experience with Databricks for data processing, analytics, and real‑time data insights.
  • Data Ingestion & Transformation:
    • Experience with Airbyte or similar tools for data ingestion.
    • dbt or other SQL‑based transformation frameworks for modular data processing.
  • Programming:
    • Proficiency in Python and/or Java/Scala for building data pipelines and custom integrations.
  • Query Languages:
    • Advanced knowledge of SQL for data manipulation and analysis.
• Soft Skills
  • Strong problem‑solving and analytical abilities.
  • Excellent communication and collaboration skills, able to effectively work in cross‑functional teams.
  • Ability to operate in a fast‑paced, agile environment and manage multiple priorities simultaneously.

Preferred Certifications & Experience

• AWS certifications (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Solutions Architect) are a plus.
• Experience with CI/CD pipelines, containerization (Docker, Kubernetes), and infrastructure as code (Terraform, CloudFormation) is beneficial.
• Familiarity with DevOps best practices for managing Airflow environments (e.g., version control for DAGs, automated testing, monitoring).
• While familiarity with financial services data, especially private equity and alternative investments, is not necessary, it will be highly impactful for this role.

WHAT WE OFFER

We are committed to supporting your development, advancing your career, and providing benefits that matter to you.

Our industry‑leading Alter Domus Academy offers six learning zones for every stage of your career, with resources tailored to your ambitions and resources from LinkedIn Learning.

Our global benefits also include

• Support for professional accreditations such as ACCA and study leave
• Flexible arrangements, generous holidays, plus an additional day off for your birthday!
• Continuous mentoring along your career progression
• Active sports, events and social committees across our offices
• 24/7 support available from our Employee Assistance Program
• The opportunity to invest in our growth and success through our Employee Share Plan

Equity in every sense of the word

We are in the business of equity, in every sense of the word. For us, this means taking action to ensure every colleague has equal opportunity, valuing every voice and experience across our organisation, maintaining an inclusive culture where you can bring your whole self to work, and making Alter Domus a workplace where everyone feels they belong.

We celebrate our differences, and understand that our success relies on diverse perspectives and experiences, working towards shared goals and a common purpose. We take pride in creating a workplace where all our people are empowered to be truly invested in the alternative and bring their whole selves to work.

We are committed to ensuring a welcoming recruiting and onboarding process for everyone. Please contact our hiring team if you require any accommodations to make our recruitment process more accessible for you.