Python Developer | Remote

Role Overview

Contract hourly role for a Software Engineer – Code QA (Remote, 10–40 hours/week). You will create realistic evaluation tasks and model responses based on real software engineering work to help assess AI reasoning and explanation quality.

Responsibilities

• Craft realistic, chat-style developer prompts covering:
  • Code review
  • Debugging
  • Error diagnosis
  • Configuration and related scenarios
• Source and adapt real pull requests and engineering discussions to produce authentic evaluation tasks.
• Write clear, technically accurate model responses that demonstrate strong reasoning and explanation quality.
• Evaluate AI outputs using logic, clarity, and technical judgment (not only whether code is executable).
• Apply software engineering expertise so prompts/responses reflect real-world developer workflows.
• Contribute to structured evaluation datasets used to measure AI reasoning and communication.
• Collaborate asynchronously with research teams to refine task quality and consistency.

Requirements

• Strong experience in software engineering, technical research, or educational content development.
• Bachelor’s degree in Software Engineering, Computer Science, or a related field.
• Strong proficiency in one or more languages including Python, JavaScript, Java, or C++.
• Hands-on experience with debugging, testing, and validating code.
• Strong technical writing skills and high attention to detail.
• Ability to work independently in a fully remote, project-based environment.

Contract & Location

• Remote (United States)
• 10–40 hours/week
• Hourly contract

Hiring Process

• Upload resume
• Interview (15 min)
• Submit form (application takes ~20 minutes)