QA/Test Engineer - Fully Remote | Upto $90/hr

QA/Test Engineer (Contract) — Fully Remote

Type: Contract (35 hours/week)
Compensation: $60–$90/hour
Location: Remote (United States)

Role Overview

As a QA/Test Engineer, you will design and execute quality checks for AI agent runs and evaluation tasks, ensuring benchmark integrity and catching edge cases and ambiguities early.

Responsibilities

• Design test cases to confirm tasks work as intended, including tricky edge cases.
• Review tasks and reference solutions to identify ambiguity and gaps early.
• Debug using Python when tasks or checks behave unexpectedly.
• Shape quality checklists and provide actionable feedback to authors.
• Protect benchmark scores by identifying shortcuts and grading gaps in AI agent runs.

Requirements

• MSc or PhD in a STEM field or equivalent practical experience.
• 1+ years in test engineering, quality assurance, or research/software engineering.
• Strong ability to design test cases, run quality-review processes, and debug complex systems.
• Proficiency in Python and Git.
• Exceptional attention to detail and clear documentation habits.

Preferred Qualifications

• Experience with AI training, model evaluation, or quality review of AI-generated work.
• A perfectionist mindset and creativity for finding overlooked issues.