Remote Quality Assurance Engineer

Role Overview

Remote Quality Assurance Engineer within the Engineering organization, responsible for quality, reliability, and release-readiness of software products. You will test applications built on modern cloud-native technologies (including Microsoft Azure) and validate AI/ML and Retrieval-Augmented Generation (RAG) features.

Responsibilities

• Own testing across the software lifecycle: test planning, execution tracking, defect management, and reporting.
• Provide release readiness input to Engineering and Product stakeholders.
• Maintain and extend the regression suite after every release.
• Conduct structured peer and code reviews of test artifacts.
• Create and maintain compliant, realistic test data, including PHI-safe synthetic healthcare data.
• Develop and maintain automated test suites using:
  • Python (e.g., pytest)
  • JavaScript/Node.js
  • Integrated into CI/CD pipelines
• Write and review SQL to validate data accuracy, completeness, and reconciliation across systems.
• Validate end-to-end data flows across source → transformation → reporting/consumption layers.
• Design and execute validation strategies for AI/ML and RAG, including:
  • model quality, grounding, and safety
  • hallucination detection and output quality
  • bias/safety checks
  • drift (regression) monitoring
• Build and maintain automated AI evaluation harnesses and datasets (e.g., prompt/response scoring, metric tracking).
• Validate applications and APIs deployed to on-prem and Microsoft Azure environments.
• Read/interpret React/Node.js/Python/ColdFusion code, pull requests, and designs to identify edge cases and regression risks.
• Write test cases/plans traced to code paths, business rules, and acceptance criteria.
• Partner with developers during reviews to flag insufficient test coverage and high-risk changes before merge.
• Perform post-deployment validation and investigate production defects with root-cause analysis.

Requirements

• 5+ years automated software testing
• 5+ years manual software testing
• 5+ years component-level testing
• 5+ years end-to-end testing
• 5+ years SQL
• 3+ years React
• 3+ years Node.js
• 3+ years Python source code in GitLab
• 2+ years testing AI/ML
• 2+ years evaluating AI/ML
• 2+ years testing RAG
• 2+ years testing generative AI features

Nice-to-haves

• Experience testing workloads on Mi (truncated in the posting text) and deeper hands-on testing of production AI systems.