Python Engineer (Remote)

Role Overview

Python Engineer (Fully Remote) to help design and evaluate autonomous AI agents across multiple LLMs and real-world domains (health, education, daily life). You will provide expert human feedback to help train and improve LLMs, including debugging agent traces and stress-testing agent behavior.

Responsibilities

• AI Agent Evaluation
  • Create evaluation rubrics with objective pass/fail criteria
  • Debug agent traces to identify failure patterns
  • Stress-test agents against edge cases, prompt injection, and tool misuse
• Technical Assessment
  • Analyze production-grade modular software architecture
  • Evaluate multi-turn system interactions and behaviors
  • Provide high-density technical feedback for LLM training
• Project Workflow
  • Create an account, upload a resume/ID, complete onboarding assessment
  • Complete flexible task assignments

Requirements

• Experience in backend engineering, AI automation, or complex systems integration
• Proven ability to build and maintain production-grade software with modular separation (e.g., distinct components/services for parsing, logic processing, reporting)
• Strong command of at least two major languages (e.g., Python, JavaScript, Go, Java)
• Experience working with SQL databases
• Practical experience building for live, non-mocked environments and handling multi-turn interactions

Nice to Have

• Experience integrating agents with live tools/APIs such as Supabase, Gmail, and other APIs
• Familiarity with persistent state and session-tracking patterns
• Experience identifying privacy leaks, authority escalation, or indirect prompt injection vulnerabilities

Compensation

• USD $30–$50/hour, paid weekly via supported payout platforms (e.g., PayPal or AirTM)