Graduate AI Test Automation Engineer

Camstgroup; Greater London; On-site; GBP 30,000 - 40,000; Full time

Job summary

A QA and software testing company in Greater London is seeking a Graduate AI Test Automation Engineer. This role entails developing Python-based automation tools and collaborating on an innovative testing framework for generative AI agents. The ideal candidate is on track to complete a degree in Computer Science or a related field and has a keen interest in AI-driven software testing. Join a cutting-edge project aimed at shaping AI testing standards, with great opportunities for career progression.

Qualifications

• On track to complete a Bachelor's or Master's degree in a relevant field.
• Strong interest in AI-driven software testing and development.
• Proficient in Python and data libraries.
• Familiarity with LLMs and testing tools would be a major asset.
• Initial experience with datasets and evaluation metrics is beneficial.

Responsibilities

• Develop and maintain Python-based test automation tools.
• Collaborate on framework design and third-party tool integration.
• Define and implement test strategies for LLM behavior validation.
• Support continuous testing, benchmarking, and KPI tracking.

Skills

Python

Data libraries

AI-driven software testing

Collaborative problem solving

Education

Bachelor's or Master's degree in Computer Science, AI, or related field (2.1 minimum grade)

Tools

PyTest

Playwright

Job description

Concept Quality Reply is a QA and software testing company focused on delivering high-quality digital solutions. We provide governance and production monitoring to ensure ongoing performance and compliance after release. Through advanced test automation and AI-driven testing strategies, we help organizations accelerate development, reduce risk and ensure reliability across the entire software lifecycle. Our goal is to transform QA into a strategic driver of innovation and efficiency.

Role Overview

We are looking for a Graduate AI Test Automation Engineer with expertise in Python and a strong interest in Artificial Intelligence who will support the development of an innovative testing framework for generative AI agents. You will be involved in designing tools for test data generation, automating evaluation processes, and contributing to AI model quality metrics (such as accuracy, bias, consistency and more). This position will allow you to join a cutting‑edge project shaping AI testing standards. You’ll work with experts in AI, testing, and data governance and will be offered opportunities for quick career progression in a high‑impact, AI‑driven environment.

Responsibilities

• Develop and maintain Python-based test automation tools
• Collaborate on framework design and third‑party tool integration
• Define and implement test strategies for LLM behavior validation
• Support continuous testing, benchmarking, and KPI tracking

About the candidate

• On track to complete (or has already completed) a Bachelor's or Master's degree in Computer Science, AI, or related field (2.1 minimum grade)
• Strong interest in AI‑driven software testing and development
• Proficient in Python and data libraries
• Familiarity with LLMs and testing tools (PyTest, Playwright, etc.) would be a major asset
• Initial experience with datasets and evaluation metrics (BLEU, ROUGE, etc.) is beneficial
• Demonstrates analytical mindset and enjoys collaborative problem solving

Reply is an Equal Opportunities Employer and committed to embracing diversity in the workplace. We provide equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type regardless of age, sexual orientation, gender, identity, pregnancy, religion, nationality, ethnic origin, disability, medical history, skin colour, marital status or parental status or any other characteristic protected by the Law.

Reply is committed to making sure that our selection methods are fair to everyone. To help you during the recruitment process, please let us know of any Reasonable Adjustments you may need.