Staff Applied AI Product Engineer

Role Overview

Staff Applied AI Product Engineer at Carrum Health. You’ll bridge state-of-the-art AI with real patient needs by architecting and integrating LLM-driven capabilities into the core product platform.

Responsibilities

• Own AI engineering strategy and lead the evolution from prototypes/demos to scalable, production-ready AI systems.
• Integrate LLM-powered features into the existing platform and evolve the stack to support AI-native workflows.
• Work with Data and Product teams (including an Applied AI PM) to:
  • Identify use cases that require LLMs vs. traditional ML
  • Build RAG (Retrieval-Augmented Generation) pipelines
  • Select foundation models
  • Iterate on prompts and design the “glue” between model intelligence and reliable product behavior
• Partner with DevOps to deploy AI services using AWS/Azure with a focus on performance, resilience, and cost-effectiveness.
• Implement monitoring and observability for high availability, security, and scalability.
• Collaborate with clinical experts to validate model outputs for safety, accuracy, and usefulness in healthcare.
• Establish patterns for smart products, model evaluation, latency optimization, and cost management.
• Ensure data privacy and HIPAA compliance and operationalize Responsible AI (fairness checks, bias detection, safety guardrails) via the AI Council.
• Mentor engineers on AI-native development practices (e.g., prompt engineering, context management).
• Partner with product leadership to define project timelines/milestones from inception to go-live and align delivery teams.

Requirements

• Hands-on leadership experience delivering LLM-driven product features (not just demos).
• Strong applied AI engineering skills, including RAG pipelines, prompt iteration, model evaluation, and productionization.
• Ability to lead cross-functional work with Data, Product, DevOps, and clinical stakeholders.
• Commitment to Responsible AI, privacy, and HIPAA-aligned practices.

Nice to Have

• Experience with vector databases and AI orchestration.
• Demonstrated expertise in latency optimization and cost management for AI systems.