Data Engineering Lead

Methodfi; Greater London; Hybrid; GBP 70,000 - GBP 90,000; Full time

Job summary

A leading financial technology firm in Greater London is seeking a Data Engineering Lead. You will own the technical roadmap for the data platform, focusing on reliability and performance. Responsibilities include delivering APIs, enhancing data ingestion, and coaching the engineering team. The ideal candidate will have hands-on experience with production data systems, strong skills in Dagster, BigQuery, and Python, and a track record of operational excellence. This role offers competitive compensation and a hybrid work model.

Benefits

Competitive Salary

Equity options

Pension matching

Private Medical Insurance

Paid sick leave

Group Life Assurance

Season Ticket Loan

Qualifications

• Hands-on experience with end-to-end production data platforms.
• Expertise in building asset-based orchestration.
• Strong SQL and BigQuery skills for performance optimization.
• Experienced with building resilient data ingestion patterns.
• Proven experience with delivering customer-facing APIs or feeds.
• Ability to write clean and testable Python code.
• Familiarity with multi-cloud environments and Infrastructure as Code.
• Experience in building observability and incident response systems.
• Strong leadership and communication skills.

Responsibilities

• Own the technical roadmap for the data platform.
• Deliver customer-facing APIs and data feeds.
• Level up orchestration using Dagster.
• Enhance data ingestion processes.
• Improve data quality and operational resilience.
• Create infrastructure as code using Terraform.
• Coach and mentor team members.

Skills

Production data platforms ownership

Dagster expertise

Advanced BigQuery skills

API and feeds experience

Clean Python coding

Multi-cloud and Terraform

Operational excellence

Clear communication skills

Tools

Dagster

Airbyte

Terraform

AWS

GCP

Job description

About 9fin

The world's largest asset class, debt, operates with the worst data.

Technology has revolutionized equity markets with electronic trading, quant algos and instantaneous news. However, in debt capital markets, the picture is completely different. It still behaves like it’s in the 1980s; trillions of dollars of trades are placed over the phone, news is slow, and corporate credit information is imperfect and scattered.

Our mission is to change this.

9fin's proprietary technology delivers fast and comprehensive financial, credit, legal & ESG analysis. Our clients are able to make faster, more informed decisions, win more business and most importantly, save time.

Our fast growing list of clients include 9 of the top 10 Investment Banks, leading Asset Managers, Hedge Funds and Law Firms.

What you’ll work on

As our Data Engineering Lead, you’ll own the technical roadmap for our data platform and raise the bar on reliability, scalability, and engineering quality. This is a hands-on, player/coach role.

• Own the Data Platform Roadmap: Set technical direction and deliver the highest-leverage platform improvements across reliability, cost, developer experience, and scale. You’ll choose, prioritise and deliver the next 12 months of platform work.

• Deliver customer-facing APIs & Feeds: Support the design and delivery of API-backed feeds and enrichment pipelines that become product features and revenue streams.

• Level Up Orchestration in Dagster: Build and refine asset-based pipelines, sensors, schedules, backfills, IO managers and monitoring patterns that are robust, idempotent and easy to operate. Define standard patterns for incremental jobs, full refreshes and reverse ETL.

• Make ingestion boring (in the best way): Improve and scale ingestion across Airbyte OSS and DLT: handle schema drift, connector health, rate limits, retries, checkpointing and operational resilience so pipelines run without heroics.

• Strengthen BigQuery foundations: Own and evolve our best practices, partitioning & clustering strategy, slot/cost management and query performance guardrails.

• Raise data quality & observability: Implement freshness SLOs, automated checks, validation, provenance and alerting so the business and customers can trust the data. Ship runbooks, incident playbooks and automated remediation where possible.

• Enable customer pipelines & reverse ETL: Own correctness, availability and SLAs for customer-facing workflows, including schema contracts and safe rollouts.

• Infrastructure as Code & CI: Own Terraform modules, CI/CD flows for both infra and data code, and deployment safety gates that prevent costly mistakes.

• Coach and grow the team: line-manage and mentor engineers, raise standards through code reviews and testing, and build a high-performance engineering culture that values operational excellence.

About You

We’re looking for a technical, high-ownership Data Engineering Lead who can ship, operate and improve production systems — and translate platform work into real product and customer outcomes.

• Hands-on Data Engineering Leader: You’ve owned production data platforms end-to-end and can lead technical direction without stepping away from the code.

• Dagster expertise: Demonstrable experience building asset-based orchestration (assets, sensors, IO managers, resources) in production.

• Warehouse & SQL depth: Advanced BigQuery skills — partitioning, clustering, materialised views, slot/cost management and query optimisation.

• Ingestion reality: You’ve operated Airbyte or similar, built resilient API/FTP/SFTP ingestion patterns, and handled rate limits, pagination, partial failures and schema evolution.

• Production API & feeds experience: You’ve shipped customer-consumable APIs or feeds (SLAs/contracts, monitoring, provenance) and can design stable downstream contracts and rollout plans.

• Python & engineering rigor: You write clean, testable Python (typing/pydantic where appropriate), author unit and integration tests, and enforce CI and linting standards.

• Multi-cloud & IaC: Comfortable with AWS (ECS/Fargate, RDS, S3) and GCP (BigQuery, GCS), and production Terraform experience.

• Operational excellence: You’ve built observability, incident response, runbooks and reduced on-call toil through automation and better design.

• Clear communicator & coach: You can explain trade-offs simply, unblock teams fast, and raise engineering quality through calm, direct leadership.

Our benefits

We’re a scaling start up, and we enjoy sharing our success, when the company succeeds, we always reinvest that in our people. We also offer huge amounts of responsibility, an abundance of opportunity for growth and a platform to truly excel.

Financial & Insurance

• Competitive Salary (our salary bands are benchmarked at the top end of the market)

• Equity options

• Pension (your minimum contributions are 4% with 9fin matching up to 7%)

• Private Medical Insurance

• Paid sick leave with Income Protection for long periods of illness

• Group Life Assurance

• Season Ticket Loan & Cycle to Work schemes

Time off

• 25 holiday days per year

• Local public holidays (with the ability to exchange them for alternative days)

• Hybrid working model, to allow you the flexibility to decide how, where and when you do your best work

• Work abroad for up to 3 months a year

• 1 month paid sabbatical after 5 years of service

• Enhanced parental leave & flexible working arrangements available

Training & Culture

• Professional learning and development budget

• Quarterly team socials

• Summer and Winter company social events

A note from the hiring manager

At 9fin, data isn’t just a support function; it’s at the core of our product. You’ll be surrounded by smart, curious people who are passionate about using data to make better decisions, improve our product, and create meaningful impact. If you’re looking for autonomy to own high-impact initiatives from conception through to delivery, and the opportunity to influence the entire organisation, then 9fin might be the perfect fit for you. I’d love to hear from you, apply now to learn more! – Rory, Analytics Engineering Manager

9fin is an equal opportunities employer

At 9fin we are dedicated to building and promoting a fair and inclusive workplace where everyone can reach their full potential and truly belong. We recognize that building diverse teams enables a more creative and productive environment. If you’re excited about this role but your experience doesn’t perfectly align with the job description, we encourage you to apply anyway. You might just be who we’re looking for — either for this role, or perhaps another.