LLM Evaluation Scenarios Architect & AI Agent Tester

Mindrift; Remote; GBP 40,000 - 60,000; Part time

Job summary

A tech consulting firm is looking for an entry-level candidate to design structured evaluation scenarios for LLM-based agents. The role is fully remote, requires a relevant degree, and invites those passionate about AI to contribute. Key responsibilities include creating test cases and defining expected agent behavior. Competitive compensation at $49/hour is offered based on expertise and experience.

Benefits

Flexible project hours

Opportunity to enhance portfolio

Competitive hourly payment

Qualifications

• Degree in Computer Science, Software Engineering, Data Science, or related fields required.
• Background in QA, software testing, or NLP annotation preferred.
• Understanding of test design principles is important.

Responsibilities

• Design structured test scenarios based on real-world tasks.
• Define the golden path and acceptable agent behavior.
• Work with developers to improve scenario clarity.

Skills

Analytical mindset

Attention to detail

Strong written communication skills

Curiosity and willingness to learn

Education

Bachelor's and/or Master's Degree in related fields

Tools

Python

JavaScript

JSON

YAML

Job description

A tech consulting firm is looking for an entry-level candidate to design structured evaluation scenarios for LLM-based agents. The role is fully remote, requires a relevant degree, and invites those passionate about AI to contribute. Key responsibilities include creating test cases and defining expected agent behavior. Competitive compensation at $49/hour is offered based on expertise and experience.