MLOps Engineer (Remote)

Role Overview

You will help design and evaluate autonomous/agentic AI systems across multiple LLMs and real-world domains (e.g., health, education, daily life). The work is coding-focused and includes providing expert human feedback to improve LLM agent performance.

Responsibilities

• AI Agent Evaluation
  • Create evaluation rubrics with objective pass/fail criteria
  • Debug agent traces to identify recurring failure patterns
  • Stress test agents with edge cases, including prompt injection and tool misuse
• Technical Assessment
  • Review and assess production-grade modular software architecture
  • Analyze multi-turn system interactions and behaviors
  • Provide high-density technical feedback to support LLM training
• Project Workflow
  • Create an account, upload a resume/ID, and complete onboarding assessment
  • Complete flexible task assignments as they are offered

Requirements

• Experience in backend engineering, AI automation, or complex systems integration
• Proven ability to build and maintain production-grade modular software (e.g., separate services for data parsing, logic processing, and reporting)
• Strong command of at least two major languages (e.g., Python, JavaScript, Go, or Java)
• Practical experience working with SQL databases
• Experience building for live, non-mocked environments and handling multi-turn interactions

Nice to Have

• Integrations with live tools/APIs (e.g., Supabase, Gmail, other APIs)
• Familiarity with persistent state and session tracking patterns
• Ability to identify privacy leaks, authority escalation, or indirect prompt injection vulnerabilities

Compensation

• Hourly: USD $30–$50, depending on experience and task complexity
• Payments issued weekly via supported payout platforms (e.g., PayPal or AirTM)