xelys jobs xelys jobs

Quality Assurance Engineer

Trilon

midpermanentqa Denver, CO 138 days ago via LinkedIn
110,000 - 125,000 USD/annual

See how well this job matches your profile

Sign up to get an AI match score and generate a tailored application in seconds.

Get your match score

Tags

Quality AssuranceTest AutomationCI/CDREST APIsAPI TestingUI TestingPythonJavaScriptLLM EvaluationRAG

About the role

Role Overview

Trilon is hiring an AI-focused Quality Assurance Engineer embedded within a product pod in Denver, CO. You will be the end-to-end owner of quality across what your pod delivers, including validating both traditional software behavior and the non-deterministic outputs of AI systems.

Responsibilities

  • Quality Ownership & Test Strategy
    • Own the full QA lifecycle: test planning through production sign-off.
    • Define acceptance criteria with engineers and product managers.
    • Create test strategies covering functionality, performance, and reliability.
    • Establish and maintain quality standards for all deliverables.
  • Test Automation & Frameworks
    • Build and maintain automated test suites for APIs, services, and UI.
    • Implement regression tests to prevent recurring issues.
    • Integrate automated tests into CI/CD pipelines for continuous validation.
    • Write production-quality, maintainable test code.
  • AI Output Evaluation
    • Design and implement evaluation for AI-generated outputs, including prompt evaluation and RAG validation.
    • Define benchmarks for accuracy, consistency, and usefulness.
    • Monitor quality drift as models, prompts, or data sources change.
    • Partner with engineers to improve output reliability and performance.
  • Defect Detection & Release Readiness (Quality Gate)
    • Identify, document, and track defects across the stack; troubleshoot with engineers.
    • Prioritize issues by impact and risk to end users.
    • Act as the final quality gate and sign-off authority for releases.
  • Collaboration & Continuous Improvement
    • Work with Lead Engineers, Applied AI Engineers, and Product Managers to improve practices.
    • Incorporate feedback from operating company users to refine quality expectations.
    • Continuously improve testing methodologies, tools, and frameworks.

Requirements

  • 4+ years in QA engineering/software testing (or related role).
  • Strong experience with test automation for APIs/backend/UI.
  • Proficiency in at least one programming language such as Python or JavaScript.
  • Experience building/maintaining automated test suites and integrating with CI/CD.
  • Experience testing APIs (e.g., REST).
  • Familiarity with modern dev practices: version control and Agile methodologies.
  • Experience testing AI-enabled applications, including LLM outputs and RAG pipelines.

Nice-to-haves

  • (Not explicitly stated beyond the listed AI and automation experience.)

About Trilon

Trilon builds technology-enabled tools for its people and partners, focused on delivering AI-powered products for real-world use. The company operates through product pods and emphasizes quality, reliability, and practical usability in its engineering workflow.

Scraped 7/8/2026