MLOps Engineer (Remote)

Role overview

You will help design and evaluate autonomous AI agents across multiple LLMs and real-world domains (health, education, daily life, and more). The work is coding-heavy and includes providing expert human feedback to improve agentic AI systems, as well as training LLMs for complex, multi-step architectural workflows.

Responsibilities

• AI Agent Evaluation
  • Write evaluation rubrics with clear, objective pass/fail criteria.
  • Debug agent traces to identify recurring failure patterns.
  • Stress test agents against edge cases, prompt injection, and tool misuse.
• Technical Assessment
  • Assess production-grade, modular software architecture.
  • Analyze multi-turn interactions and behavioral patterns.
  • Provide high-density technical feedback to support LLM training.
• Project workflow
  • Create an account, upload resume/ID, complete onboarding assessment.
  • Complete flexible, assigned tasks to earn compensation.

Requirements

• Experience in backend engineering, AI automation, or complex systems integration.
• Proven ability to build and maintain production-grade modular software (e.g., separated services for data parsing, logic processing, and reporting).
• Strong command of at least two major languages (e.g., Python, JavaScript, Go, Java).
• Experience working with SQL databases.
• Practical experience building for live, non-mocked environments and handling multi-turn system interactions.

Nice to have

• Integrating agents with live tools/APIs such as Supabase and Gmail.
• Familiarity with persistent state and session-tracking patterns.
• Ability to identify privacy leaks and vulnerabilities such as authority escalation and indirect prompt injection.

Compensation

• USD $30–$50/hour, weekly payments via supported payout platforms (e.g., PayPal or AirTM).