Principal Platform Engineer

London Stock Exchange Group; City of Edinburgh; On-site; GBP 70,000 - GBP 90,000; Full time

Job summary

A global financial markets infrastructure company is seeking a Principal Platform Engineer to enhance a cloud-native data platform in Edinburgh. You will leverage technologies like Azure, Databricks, and Kafka to manage data pipelines for scalable analytics. Successful candidates will have over 5 years of hands-on experience with Azure and a background in data-driven environments. This role offers the opportunity to influence technical direction and contribute to innovative solutions in a collaborative team atmosphere.

Benefits

Healthcare

Retirement planning

Paid volunteering days

Wellbeing initiatives

Qualifications

• 5–10+ years of experience in building and operating scalable systems in production.
• Hands-on experience with Azure and Databricks for data workloads.
• Familiarity with Kafka, event-driven architectures and SQL.

Responsibilities

• Work on a cloud-native data platform for real-time and batch processing.
• Collaborate with teams to optimize query performance on large datasets.
• Ensure reliability, observability, and secure delivery of the platform.

Skills

Azure

Apache Spark

Data pipelines

SQL

Python

Scala

Kafka

Tools

Microsoft Azure

Databricks

ADLS Gen2

Cosmos DB

Job description

Principal Platform Engineer page is loaded## Principal Platform Engineerlocations: GBR-Edinburgh-5th Floor, Exchange Towertime type: Full timeposted on: Posted Todayjob requisition id: R0115985LSEG (London Stock Exchange Group) is more than a diversified global financial markets infrastructure and data business. We are dedicated, open-access partners with a dedication to excellence in delivering the services our customers expect from us. With extensive experience, deep knowledge and worldwide presence across financial markets, we enable businesses and economies around the world to fund innovation, manage risk and create jobs. It’s how we’ve contributed to supporting the financial stability and growth of communities and economies globally for more than 300 years. Through a comprehensive suite of trusted financial market infrastructure services – and our open-access model – we provide the flexibility, stability and trust that enable our customers to pursue their ambitions with confidence and clarity.LSEG is headquartered in the United Kingdom, with significant operations in 70 countries across EMEA, North America, Latin America and Asia Pacific. We employ 25,000 people globally, more than half located in Asia Pacific. LSEG’s ticker symbol is LSEG.## **OUR PEOPLE:**People are at the heart of what we do and drive the success of our business. Our culture of connecting, creating opportunity and delivering excellence shape how we think, how we do things and how we help our people fulfil their potential. We embrace diversity and actively seek to attract individuals with unique backgrounds and perspectives. We break down barriers and encourage teamwork, enabling innovation and rapid development of solutions that make a difference. Our workplace generates an enriching and rewarding experience for our people and customers alike. Our vision is to build an inclusive culture in which everyone feels encouraged to fulfil their potential.We know that real personal growth cannot be achieved by simply climbing a career ladder – which is why we encourage and enable a wealth of avenues and interesting opportunities for everyone to broaden and deepen their skills and expertise. As a global organisation spanning 70 countries and one rooted in a culture of growth, opportunity, diversity and innovation, LSEG is a place where everyone can grow, develop and fulfil your potential with meaningful careers!!## **ROLE SUMMARY:**You will work on a modern, cloud‑native data platform built on **Microsoft Azure**, and you’ll have substantial input into how the platform evolves over time. The environment supports large‑scale, business‑critical workloads and is developed for both real‑time and batch processing.The platform makes extensive use of **Azure Databricks**, Apache Spark, and Delta Lake to support high‑throughput data pipelines. **Kafka and Azure Event Hubs** are used to implement reliable, event‑driven architectures, with schema management and stream processing forming core parts of the system.Datasets are stored and served primarily using **ADLS Gen2 and Delta tables**, with additional technologies such as Azure Data Explorer, Cosmos DB, or Synapse used where they are the right fit. Engineers work closely with **SQL**, Python, and Scala to model data, optimise query performance, and support high‑QPS analytical use cases.The platform places a strong emphasis on **reliability, observability, and secure delivery**. Teams use Datadog and Azure Monitor, follow Open Telemetry standards, and handle environments using Git‑based workflows and Infrastructure as Code. Automated testing and well‑defined release strategies are central to how changes are built and shipped.## **WHAT YOU'LL BRING:**You will bring 5–10+ years of experience building and operating highly scalable systems in production, ideally within data‑intensive or analytics‑heavy environments. You’ve worked hands‑on with Azure and have experience operating Databricks platforms supporting both streaming and batch workloads.You will have practical experience working with Kafka or Azure Event Hubs, including designing partitions, managing consumer groups, and understanding delivery semantics. You’re confident working with SQL, have experience with data modelling, and know how to diagnose and improve query performance on large datasets.A proven understanding of distributed systems fundamentals, such as fault tolerance, stateful processing, and system resilience. You’ve worked with event‑driven architectures and have supported systems that handle high request volumes and low‑latency analytical workloads.You’re comfortable working in at least one data‑focused programming language, such as Python or Scala, and you bring a good sense of ownership. You enjoy influencing technical direction, mentoring other specialists, and helping teams converge on shared standards and guidelines.## **DESIRABLE'S:**While not required, experience in some of the following areas would be advantageous. This could include working with **Delta Live Tables, Unity Catalog**, or broader Lakehouse governance approaches, or exposure to additional data and streaming technologies such as **Flink, Azure Data Explorer (Kusto), or Cosmos DB**.Experience crafting serving layers, stream‑out patterns, or **feature stores for analytics or machine learning** is also valuable. Familiarity with **SRE practices**, such as defining SLOs and handling error budgets in data‑intensive systems, is a plus.**Career Stage:**Manager**London Stock Exchange Group (LSEG) Information:**Join us and be part of a team that values innovation, quality, and continuous improvement. If you're ready to take your career to the next level and make a significant impact, we'd love to hear from you.LSEG is a leading global financial markets infrastructure and data provider. Our purpose is driving financial stability, empowering economies and enabling customers to create sustainable growth.Our purpose is the foundation on which our culture is built. Our values of **Integrity, Partnership**, **Excellence** and **Change** underpin our purpose and set the standard for everything we do, every day. They go to the heart of who we are and guide our decision making and everyday actions.Working with us means that you will be part of a dynamic organisation of 25,000 people across 65 countries. However, we will value your individuality and enable you to bring your true self to work so you can help enrich our diverse workforce.We are proud to be an equal opportunities employer. This means that we do not discriminate on the basis of anyone’s race, religion, colour, national origin, gender, sexual orientation, gender identity, gender expression, age, marital status, veteran status, pregnancy or disability, or any other basis protected under applicable law. Conforming with applicable law, we can reasonably accommodate applicants' and employees' religious practices and beliefs, as well as mental health or physical disability needs.You will be part of a collaborative and creative culture where we encourage new ideas. We are committed to sustainability across our global business and we are proud to partner with our customers to help them meet their sustainability objectives. Our charity, the LSEG Foundation provides charitable grants to community groups that help people access economic opportunities and build a secure future with financial independence. Colleagues can get involved through fundraising and volunteering.LSEG offers a range of tailored benefits and support, including healthcare, retirement planning, paid volunteering days and wellbeing initiatives.Please take a moment to read this carefully, as it describes what personal information London Stock Exchange Group (LSEG) (we) may hold about you, what it’s used for, and how it’s obtained, .If you are submitting as a Recruitment Agency Partner, it is essential and your responsibility