Go/Golang Developer LLM - Remote

Role Overview

You will work with an AI evaluation project to assess and improve how conversational AI models reason about code, generate programming solutions, and explain technical concepts.

Responsibilities

• Evaluate LLM-generated responses to coding and software engineering questions for:
  • accuracy, reasoning quality, clarity, completeness
• Fact-check outputs using trusted public sources and authoritative references
• Run accuracy tests by executing code and validating outputs with appropriate tools
• Annotate model responses by identifying:
  • strengths, areas for improvement, and factual/conceptual inaccuracies
• Assess code quality including readability, algorithmic soundness, and explanation quality
• Ensure responses follow expected conversational behavior and system guidelines
• Apply consistent evaluation standards using provided taxonomies, benchmarks, and detailed guidelines

Requirements

• BS, MS, or PhD in Computer Science or a closely related field
• Significant real-world experience in software engineering or related technical roles
• Expert in at least one programming language, including Go/Golang (also acceptable: Python, Java, C++, JavaScript, Rust)
• Ability to solve HackerRank or LeetCode Medium and Hard problems independently
• Experience contributing to well-known open-source projects, including merged pull requests
• Strong understanding of LLMs while coding, including strengths and failure modes
• Strong attention to detail; comfortable evaluating complex technical reasoning and identifying subtle bugs/logical flaws

Nice-to-Haves

• Experience with RLHF, model evaluation, or data annotation
• Competitive programming track record
• Experience reviewing code in production environments
• Familiarity with multiple programming paradigms/ecosystems
• Ability to explain complex technical concepts to non-experts

Success Criteria

• Identify incorrect logic, inefficiencies, edge cases, or misleading explanations
• Improve the correctness, robustness, and clarity of AI coding outputs
• Deliver reproducible evaluation artifacts that strengthen model performance