Quality Assurance Engineer
Trilon
midpermanentqa Denver, CO 138 days ago via LinkedIn
110,000 - 125,000 USD/annual
See how well this job matches your profile
Sign up to get an AI match score and generate a tailored application in seconds.
Get your match scoreTags
Quality AssuranceTest AutomationCI/CDREST APIsAPI TestingUI TestingPythonJavaScriptLLM EvaluationRAG
About the role
Role Overview
Trilon is hiring an AI-focused Quality Assurance Engineer embedded within a product pod in Denver, CO. You will be the end-to-end owner of quality across what your pod delivers, including validating both traditional software behavior and the non-deterministic outputs of AI systems.
Responsibilities
- Quality Ownership & Test Strategy
- Own the full QA lifecycle: test planning through production sign-off.
- Define acceptance criteria with engineers and product managers.
- Create test strategies covering functionality, performance, and reliability.
- Establish and maintain quality standards for all deliverables.
- Test Automation & Frameworks
- Build and maintain automated test suites for APIs, services, and UI.
- Implement regression tests to prevent recurring issues.
- Integrate automated tests into CI/CD pipelines for continuous validation.
- Write production-quality, maintainable test code.
- AI Output Evaluation
- Design and implement evaluation for AI-generated outputs, including prompt evaluation and RAG validation.
- Define benchmarks for accuracy, consistency, and usefulness.
- Monitor quality drift as models, prompts, or data sources change.
- Partner with engineers to improve output reliability and performance.
- Defect Detection & Release Readiness (Quality Gate)
- Identify, document, and track defects across the stack; troubleshoot with engineers.
- Prioritize issues by impact and risk to end users.
- Act as the final quality gate and sign-off authority for releases.
- Collaboration & Continuous Improvement
- Work with Lead Engineers, Applied AI Engineers, and Product Managers to improve practices.
- Incorporate feedback from operating company users to refine quality expectations.
- Continuously improve testing methodologies, tools, and frameworks.
Requirements
- 4+ years in QA engineering/software testing (or related role).
- Strong experience with test automation for APIs/backend/UI.
- Proficiency in at least one programming language such as Python or JavaScript.
- Experience building/maintaining automated test suites and integrating with CI/CD.
- Experience testing APIs (e.g., REST).
- Familiarity with modern dev practices: version control and Agile methodologies.
- Experience testing AI-enabled applications, including LLM outputs and RAG pipelines.
Nice-to-haves
- (Not explicitly stated beyond the listed AI and automation experience.)
About Trilon
Trilon builds technology-enabled tools for its people and partners, focused on delivering AI-powered products for real-world use. The company operates through product pods and emphasizes quality, reliability, and practical usability in its engineering workflow.
Scraped 7/8/2026