Freelance Software Engineer (Remote)

Role Overview

Freelance Software Engineer (Fully Remote) helping design and evaluate autonomous AI agents across multiple LLMs. You’ll provide expert human feedback for LLM training and help assess complex, multi-step agent workflows in domains such as health, education, and everyday life.

Responsibilities

• AI Agent Evaluation
  • Create evaluation rubrics with objective pass/fail criteria
  • Debug agent traces to identify failure patterns
  • Stress test agents for edge cases, prompt injection, and tool misuse
• Technical Assessment
  • Review and analyze production-grade modular software architecture
  • Evaluate multi-turn system interactions and behaviors
  • Provide high-density technical feedback to support LLM training
• Project Workflow
  • Create an account and upload a resume/ID
  • Complete the onboarding assessment
  • Complete flexible task assignments and begin earning

Requirements

• Backend engineering experience in AI automation, complex systems integration, or similar
• Proven ability to build and maintain production-grade modular software (e.g., distinct services for data parsing, logic processing, reporting)
• Strong command of at least two major languages (e.g., Python, JavaScript, Go, Java)
• Experience working with SQL databases
• Practical experience building for live, non-mocked environments
• Ability to work with multi-turn system interactions

Nice to Have

• Integrating agents with live tools/APIs such as Supabase and Gmail
• Familiarity with persistent state and session-tracking patterns
• Experience identifying security/privacy risks (e.g., privacy leaks, authority escalation, indirect prompt injection vulnerabilities)

Compensation

• $30–$50 USD/hour, depending on experience and task complexity
• Weekly payments via supported payout platforms (e.g., PayPal or AirTM)
• Full compensation details provided prior to task acceptance