AI Engineer

Role: AI Engineer

Build and ship production AI agents for Kobie’s internal agent platform.

What you’ll build

• Agent harnesses on Amazon AgentCore that automate analyst workflows
• Tools/skills agents call (including integrations and Snowflake-backed retrieval)
• Evaluation and reliability for what makes it to production

Responsibilities

• Agent Development
  • Build agent harnesses in Python using LangChain and LangGraph (tool-calling, structured outputs with Pydantic/JSON schema, retries, streaming, memory)
  • Package harnesses for the AgentCore runtime with the right context, tools, skills, and subagents for production flows
  • Write the tools/skills agents use, including:
    • API integrations
    • SQL queries and Snowflake retrieval
    • Knowledge retrieval with clear contracts and Pydantic validation
• Evaluation and Reliability
  • Create evaluation harnesses (e.g., golden datasets, LLM-as-judge, regression suites) using AgentCore Evaluations
  • Integrate evaluation into CI
  • Add guardrails for tool execution:
    • auth scoping
    • input/output validation
    • PII and prompt-injection protections
    • hallucination mitigation
  • Own the full cycle: prototype, deploy via Amazon AgentCore, monitor traces, and fix production issues
• Collaboration
  • Partner with data engineers on Snowflake-backed retrieval patterns (e.g., Cortex Analyst, Cortex Search Services)
  • Contribute to evolving internal engineering patterns

Requirements

• 3+ years professional Python with experience building and operating services in production
• 1+ years hands-on LLM work in production: prompt/context engineering, tool/function calling, structured outputs, and RAG
• Working knowledge of LangChain/LangGraph (or comparable frameworks: AgentCore, CrewAI, Semantic Kernel)
• Experience with LLM observability tools such as CloudWatch, LangSmith, Langfuse, MLflow, or OpenTelemetry
• Experience designing evaluation frameworks (e.g., MLflow, DeepEval, LLM-as-judge, multi-turn regression)
• Fluency with Git, Docker, and modern API frameworks
• Strong written communication and good judgment about readiness to ship

Notes

• This is not a research role: you’ll prototype, ship, monitor, and iterate on features used by real teams.
• A bachelor’s degree is not required; equivalent practical experience is accepted.