Head Of Product

Role Overview

Head of Product – Model Evaluation at Applause (full remote). You will lead the development of a strategic new AI evaluation platform, defining the product vision and shaping the go-to-market strategy while reporting directly to the CTO.

Key Missions

• Define the product vision and establish target customer personas and product positioning.
• Shape the go-to-market strategy, including messaging and early adoption.
• Collaborate with Sales and Marketing to validate demand, refine messaging, and drive early adoption.
• Act as the bridge across Product, Data Science, Engineering, and Commercial teams.
• Build foundational product capabilities such as:
  • LLM-as-a-judge systems
  • Model observability frameworks

Requirements

• Ability to engage technically with ML engineers (discuss model architectures, evaluation metrics, and API design).
• Proven experience owning or contributing to 0→1 product development and go-to-market.
• 5+ years of product management experience, ideally with AI/ML-driven or data products.
• Demonstrated ability to take a product from concept to paying customers.
• Strong skills in customer personas, value propositions, and positioning.
• Strong understanding of metrics, experimentation, and data quality.
• Experience working closely with data science / machine learning teams.
• Background in data science, machine learning, or engineering.
• Experience with model evaluation systems, including human and/or automated approaches.
• Familiarity with B2B/enterprise products and technical buyer personas.

Nice-to-haves

• Familiarity with LLM-as-a-judge, pairwise ranking, or preference modeling.
• Understanding of limitations and tradeoffs in evaluating generative AI systems.
• Experience with model evaluation/tooling companies (e.g., Arize, Langsmith, Galileo, Scale AI, Weights & Biases, Humanloop) or as an enterprise buyer/implementer.
• MBA or equivalent experience.
• Experience in high-growth, product-led organizations.