Job description
Job details
- Location: Manchester, Greater Manchester
- Work mode: Remote
- Employment type: Freelance (Not an internship)
- Salary: GBP 67,833 per year
Role overview
Mindrift is seeking a Freelance Agent Evaluation Engineer to join a specialized project focused on enhancing AI coding agents. In this role, you will contribute to the development of high-quality datasets used to test how AI models handle complex, real-world developer tasks. You will be responsible for designing challenging scenarios and defining strict evaluation criteria to ensure AI systems meet professional software engineering standards.
Job details
This is a Freelance, project-based position located in Manchester, Greater Manchester. This role is not an internship. The work mode is Remote, allowing you to contribute to leading tech projects from your own environment. The offered salary for this engagement is 67,833 GBP.
Responsibilities
- Create challenging real-world developer tasks to evaluate AI coding agents.
- Develop detailed evaluation criteria and benchmarks for model performance.
- Analyze AI-generated code for accuracy, efficiency, and security.
- Collaborate with technical teams to refine dataset quality and diversity.
- Provide expert feedback on AI model behavior in complex coding scenarios.
Requirements
- Proven experience in software development and professional coding practices.
- Strong proficiency in English, both written and verbal.
- Ability to design complex technical problems and edge cases for AI testing.
- Experience with AI tools or Large Language Models (LLMs) is highly preferred.
- Strong analytical skills to assess the quality of AI-generated technical solutions.
Benefits
- Flexible freelance work arrangements.
- Opportunity to work with leading global tech companies.
- Engagement in cutting-edge AI development and evaluation.
Keywords
AI EvaluationLLM TestingSoftware EngineeringCoding AgentsDataset CreationQuality AssurancePythonPrompt Engineering