Senior Machine Learning Engineer: Search Quality

Role overview

As a Senior Machine Learning Engineer (Search Quality) on the Search Quality team at Constructor, you will improve ecommerce search relevance for hundreds of millions of users. The team’s mission is to measure search quality, raise it consistently, and detect degradations before users notice.

Responsibilities

• LLM-based relevance evaluation
  • Fine-tune models to assess relevance and align judgments with real user behavior
  • Teach models to understand query intent and represent items from messy catalog data
• Real-time quality improvements in production
  • Build systems for reranking, filtering, and signal computation
  • Manage quality vs latency tradeoffs (latency is a strict requirement)
• Automated quality monitoring & insights
  • Create pipelines to detect degradations and identify underperforming patterns
  • Develop agentic systems that generate actionable recommendations for product data and search configurations
• Drive impact through experimentation and collaboration with cross-functional teams (ranking, product, data engineering)

Requirements

• 4+ years shipping production ML systems
• Experience with search / information retrieval / recommendation systems
• Hands-on experience with fine-tuning, evaluation frameworks, and scaling LLM deployments
• Strong Python and PyTorch
• Fluency in SQL and data orchestration tools: Spark and Airflow
• Experience designing and running A/B tests to validate model impact
• Excellent English communication and cross-functional collaboration

Nice to haves / Context

• Ability to generalize across 40+ languages and 20+ domains without per-customer rules
• Experience operating without a single universal notion of ground truth (relevance can vary by retailer type)