Data Science Expert

Role overview

You will work as a Data Science Expert to evaluate how effectively AI completes real-world data science tasks. Instead of building models or producing business analyses yourself, you will define what “high-quality” work looks like and assess AI-generated (and human) outputs using evidence-based evaluation frameworks.

Key responsibilities

• Design task-specific evaluation criteria for real-world deliverables, including:
  • Data analyses
  • Predictive and statistical models
  • Dashboards and visualizations
  • Experiment results
  • Executive recommendations and business insights
• Evaluate AI-generated and human-produced work using objective scoring frameworks
• Write detailed, consistent justifications for each evaluation, ensuring technical accuracy
• Use analytical judgment to produce reproducible and defensible assessments
• Incorporate structured reviewer feedback and continuously refine evaluation methodologies
• Collaborate cross-functionally to improve AI training quality and evaluation standards

Required qualifications

• 5+ years of professional data science experience in an industry environment
• Strong background in business operations, product analytics, growth analytics, or similar functions at technology-driven organizations
• Deep expertise in:
  • Experimental design and A/B testing
  • Statistical analysis
  • SQL and Python
  • Metric development and performance measurement
  • Translating analytical findings into business recommendations
• Excellent written communication and the ability to clearly explain technical reasoning
• High attention to detail and consistent evaluation against structured rubrics
• Comfortable receiving feedback and calibrating evaluation standards with peer reviewers

Preferred qualifications

• Experience with AI evaluation, model training, human feedback, or data annotation initiatives
• Familiarity with LLMs, AI-assisted analytics, or machine learning workflows
• Experience reviewing technical work or mentoring analytical teams

Contract details

• Independent contractor engagement
• Fully remote with flexible working hours
• Weekly payments based on approved completed work
• Project duration may extend or end based on needs and performance
• H1-B and STEM OPT sponsorship not available