AI QA Specialist

Hitachi Digital LLC; Birmingham; On-site; GBP 60,000 - GBP 70,000; Full time

Job summary

A leading technology firm in Birmingham is seeking an AI QA Specialist to design and execute test strategies for AI systems. The ideal candidate will validate model outputs and ensure compliance with governance standards. Responsibilities include developing QA frameworks and building automated test suites. Candidates should have a strong programming background in Python and at least five years of experience in QA, specifically with AI and ML systems. This role emphasizes collaboration and continuous improvement.

Benefits

Flexible work arrangements

Industry-leading benefits

Support for holistic health and wellbeing

Qualifications

• 5+ years in QA or testing; 2+ years with AI/ML systems.
• Hands-on experience with LLMs and Agentic AI frameworks.
• Familiar with data governance frameworks and compliance standards.

Responsibilities

• Design and execute test strategies for AI systems.
• Validate AI model outputs for accuracy and compliance.
• Build automated test suites for AI workflows.

Skills

Python programming

AI/ML systems

Test automation tools

Analytical skills

Collaboration

Education

Bachelor’s or Master’s in Computer Science, AI/ML

Tools

GCP services (Vertex AI, IAM)

Evaluation harnesses

Job description

Overview

We’re Hitachi Digital, a company at the forefront of digital transformation and the fastest growing division of Hitachi Group. We’re crucial to the company’s strategy and ambition to become a premier global player in the massive and fast-moving digital transformation market.

Our group companies, including GlobalLogic, Hitachi Digital Services, Hitachi Vantara and more, offer comprehensive services that span the entire digital lifecycle, from initial idea to full-scale operation and the infrastructure to run it on. Hitachi Digital represents One Hitachi, integrating domain knowledge and digital capabilities, and harnessing the power of the entire portfolio of services, technologies, and partnerships, to accelerate synergy creation and make real-world impact for our customers and society as a whole.

Imagine the sheer breadth of talent it takes to unleash a digital future. We don’t expect you to ‘fit’ every requirement – your life experience, character, perspective, and passion for achieving great things in the world are equally as important to us.

The Team

As an AI QA Specialist, you will design and execute test strategies for AI systems, including LLMs, Agentic AI frameworks, and RAG pipelines. You will validate model outputs, monitor AI observability, and enforce governance and compliance standards. Your role ensures that AI solutions meet functional and non-functional requirements before deployment.

The role

AI Testing & Validation

• Develop QA frameworks for AI models, multi-agent systems, and orchestration workflows.
• Design and execute functional, regression, and performance tests for AI pipelines.
• Validate LLM outputs for accuracy, bias, and compliance with enterprise standards.

Observability & Explainability

• Implement AI observability dashboards for monitoring latency, reliability, and error rates.
• Ensure explainability of AI decisions through evaluation harnesses and reporting tools.
• Conduct bias detection, drift monitoring, and safety checks for deployed models.

Automation & Integration

• Build automated test suites for AI workflows using Python and CI/CD pipelines.
• Integrate QA processes with GCP services (Vertex AI, Cloud Run, GKE) for continuous validation.
• Collaborate with engineering teams to embed QA checkpoints into development lifecycle.

Governance & Compliance

• Enforce AI safety protocols, data privacy standards, and regulatory compliance.
• Document test plans, evaluation metrics, and audit-ready reports.

What you’ll bring

• Bachelor’s or Master’s in Computer Science, AI/ML, or related field.
• 5+ years in QA or testing; 2+ years with AI/ML systems.
• Hands-on experience with LLMs, Agentic AI frameworks, and RAG pipelines.
• Strong programming skills in Python; familiarity with test automation tools.
• Knowledge of GCP services (Vertex AI, IAM, VPC-SC) and cloud-native QA practices.
• Understanding of AI observability, explainability frameworks, and bias detection.

Preferred Qualifications

• Certifications: Google Professional Cloud Architect, AI/ML Testing Specialist.
• Experience with LangChain, evaluation harnesses, and model monitoring tools.
• Familiarity with data governance frameworks and compliance standards.
• Exposure to GPU optimization and performance benchmarking.
• Strong analytical and problem-solving skills.
• Ability to design robust QA strategies for complex AI systems.
• Collaborative mindset with excellent communication skills.
• Passion for responsible AI, quality assurance, and continuous improvement.

Success Metrics

• Deployment of comprehensive QA frameworks for AI workflows.
• Reduction in model errors, bias, and drift incidents.
• Improved AI reliability and compliance readiness.
• Positive impact on user trust and operational efficiency.

What You’ll Work With

• AI Platforms: Vertex AI, Gemini Enterprise, Agentic AI frameworks.
• Testing Tools: Python-based automation, CI/CD pipelines, evaluation harnesses.
• Observability & Governance: Dashboards, explainability tools, compliance frameworks.

About us

We’re a global, 1000-strong, diverse team of professional experts, promoting and delivering Social Innovation through our One Hitachi initiative (OT x IT x Product) and working on projects that have a real-world impact. We’re curious, passionate and empowered, blending our legacy of 110 years of innovation with our shaping our future. Here you’re not just another employee; you’re part of a tradition of excellence and a community working towards creating a digital future.

We are committed to fostering an inclusive culture based on mutual respect and merit-based systems. We believe that when people feel valued, heard, and safe to express themselves, they do their best work.

How we look after you

We help take care of your today and tomorrow with industry-leading benefits, support, and services that look after your holistic health and wellbeing. We’re also champions of work-life balance and offer flexible arrangements that work for you (role and location dependent). We’re always looking for new ways of working that bring out our best, which leads to unexpected ideas. So here, you’ll experience a sense of belonging, and discover autonomy, freedom, and ownership as you work alongside talented people you enjoy sharing knowledge with.

We’re proud to say we’re an equal opportunity employer and welcome all applicants for employment without regard to race, colour, religion, sex, sexual orientation, gender identity, national origin, veteran status, age, disability status or any other protected characteristic. Should you need reasonable accommodations during the recruitment process, please let us know so that we can do our best to set you up for success.