Job description
Job details
- Location: Birmingham, West Midlands
- Work mode: Remote
- Employment type: Freelance (Not an internship)
- Salary: GBP 67,501 per year
Role overview
Mindrift is seeking a skilled Freelance Agent Evaluation Engineer based in Birmingham, West Midlands, to help refine the capabilities of next-generation AI coding agents. In this role, you will design complex developer tasks and establish rigorous evaluation criteria to measure how effectively AI models handle real-world software engineering challenges. This is a freelance, project-based opportunity focused on the intersection of software development and AI quality assurance.
Job details
This position is located in Birmingham, West Midlands, and operates on a Remote basis. This is a Freelance contract and is not an Internship. The offered salary for this project is 67,501 GBP.
Responsibilities
- Develop challenging real-world coding tasks to test the limits of AI agents
- Create detailed evaluation criteria and benchmarks for AI-generated code
- Analyze model outputs to identify failures in logic or implementation
- Build high-quality datasets used to train and refine AI coding capabilities
- Collaborate with tech teams to improve AI system performance and reliability
Requirements
- Proven experience in software development and engineering practices
- Strong proficiency in English, both written and verbal
- Ability to decompose complex developer workflows into testable tasks
- Experience with AI models or LLM evaluation is highly preferred
- Detail-oriented approach to creating technical documentation and criteria
Benefits
- Flexible remote work environment
- Opportunity to work with leading tech companies
- Engagement in cutting-edge AI development projects
- Competitive project-based compensation
Keywords
AI EvaluationLLMCoding AgentsSoftware TestingDataset CreationPythonPrompt EngineeringQuality Assurance