Job description
Job details
- Location: Glasgow, Scotland
- Work mode: Remote
- Employment type: Freelance (Not an internship)
- Salary: GBP 66,355 per year
Role overview
Mindrift is seeking a Freelance Agent Evaluation Engineer to help build high-quality datasets for evaluating AI coding agents. In this role, you will design challenging real-world developer tasks and establish rigorous evaluation criteria to measure how effectively AI models handle complex programming scenarios. This is a project-based freelance opportunity designed for specialists who want to contribute to the improvement of leading AI systems.
Job details
This freelance position is based in Glasgow, Scotland, and operates as a remote-friendly project. This is not an internship. The compensation for this engagement is 66,355 GBP.
Responsibilities
- Develop complex, real-world coding tasks to test the capabilities of AI agents.
- Create detailed evaluation criteria and rubrics to measure model performance.
- Analyze AI-generated code for accuracy, efficiency, and adherence to best practices.
- Collaborate with AI research teams to refine dataset quality and diversity.
- Document edge cases and failure modes observed during the evaluation process.
Requirements
- Proven experience in software development with proficiency in multiple programming languages.
- Strong understanding of AI model behavior and the challenges of automated coding.
- Ability to design rigorous test cases that challenge current LLM capabilities.
- Professional level of English proficiency for technical documentation and communication.
- Experience working independently in a project-based or freelance capacity.
Benefits
- Opportunity to work with leading tech companies on cutting-edge AI projects.
- Flexible project-based schedule.
- Competitive freelance compensation.
Keywords
AI EvaluationLLM TestingCoding AgentsDataset CreationSoftware EngineeringPrompt EngineeringQuality Assurance