Job description
Job details
- Location: Australia
- Work mode: Remote
- Employment type: Freelance (Not an internship)
- Salary: Salary details are available in the employer description.
Role overview
Mindrift is seeking a Freelance Agent Evaluation Engineer to contribute to project-based AI evaluation initiatives across Australia. This remote, freelance position focuses on testing and improving AI coding agents by creating challenging developer tasks and evaluation criteria. The role requires strong English proficiency and experience in software development and AI systems assessment.
Job details
This is a freelance, project-based opportunity available remotely throughout Australia. Compensation details are available in the employer description. You will work on building datasets to evaluate how AI models handle real-world developer tasks, contributing to the advancement of AI coding agent capabilities for leading tech companies.
Responsibilities
- Create challenging tasks to evaluate AI coding agent performance
- Develop evaluation criteria for real-world developer scenarios
- Test and assess AI system capabilities on coding tasks
- Build datasets for AI model evaluation and improvement
- Document evaluation methodologies and results
Requirements
- Strong English proficiency (written and spoken)
- Experience in software development and coding practices
- Understanding of AI systems and machine learning concepts
- Ability to design realistic developer task scenarios
- CV submitted in English required
Benefits
- Flexible project-based work schedule
- Remote work from anywhere in Australia
- Collaborate with leading tech companies
- Gain experience in AI evaluation
Keywords
AI evaluationmachine learningcoding agentsdataset creationsoftware testingAI systemsdeveloper tasksmodel evaluation