Senior Product Quality Analyst - AI Voice

Role Overview

Senior Product Quality Analyst – AI Voice (Personalization / Voice Production Team)

As a Senior Product Quality Analyst, you will bring an evaluation and quality perspective to Spotify’s AI voice experiences. You’ll help define and measure what “AI voice quality” means across multiple products, audiences, languages, and use cases—answering whether Spotify’s AI voice technology is the best for Spotify’s specific needs.

What You’ll Do

• Own the quality bar for Spotify’s AI voice models, defining what “good enough” means for different products and markets.
• Design evaluation rubrics and build an evaluation approach to measure qualities such as:
  • naturalness, tone, pacing
  • pronunciation, accent
  • audio quality and overall experience
• Plan and run evaluation campaigns, translating qualitative and quantitative findings into clear recommendations for Voice Production, Speak, and product teams.
• Establish an ongoing evaluation rhythm (when and how model quality should be reassessed as technology and products evolve).
• Benchmark Spotify’s AI voices against third-party models and competitors to identify strengths, differences, and opportunities to improve.
• Recommend quality trade-offs as Spotify expands into additional languages and regions.
• Partner with voice experts, ML teams, product partners, and vendors to maintain a consistent quality bar under competing priorities.
• Surface meaningful gaps in model performance and use evaluation insights to influence future research and technical priorities.

Who You Are

• Experience in product quality, evaluation, data curation, or a related area involving human judgment to assess model/product performance.
• Ability to independently define evaluation criteria and convert subjective qualities into clear, repeatable rubrics.
• Comfortable designing and running qualitative and MOS-style evaluations and synthesizing results into actionable insights.
• Understanding (or ability to quickly develop expertise) in voice/audio quality (naturalness, pacing, tone, accent, pronunciation).
• Strong judgment to maintain a consistent quality bar while balancing requirements across products, markets, and stakeholders.
• Ability to work autonomously: identify what must be evaluated and set evaluation processes and cadence.
• Clear communication across creative, technical, and product teams; comfortable acting as a point of contact for partners and vendors.
• Curiosity about generative AI and speech technology, and ability to translate industry changes into relevant benchmarks.

Nice-to-Haves (Implied)

• Experience evaluating AI voice/speech systems in multilingual or multi-market contexts.
• Familiarity with comparing in-house models to third-party competitors.