AI Engineer (LLM/RAG) (m/w/d)

Role: AI Engineer (LLM/RAG)

Nejo is looking for an experienced AI Engineer to take ownership of production LLM/RAG systems, improve reliability and quality, and extend them to new use cases. The role is applied LLM engineering combined with solid backend engineering in TypeScript.

Responsibilities

• Maintain and improve existing LLM pipelines
  • Assess current architecture and failure modes
  • Prioritize fixes and refactor/extend without disrupting production
• Own RAG systems end-to-end
  • Document ingestion & parsing, chunking, indexing
  • Hybrid retrieval (BM25 + vector), query rewriting
  • Reranking and grounded generation with citations
• Implement retrieval and access controls
  • Chunk-level access control
  • Index freshness, tenant isolation across retrieval systems
• Build content generation pipelines
  • Consistent quality at volume, including human review steps
• Develop and operate automated workflows
  • Work with internal and third-party systems (e.g., ERP, CRM, email, internal APIs)
  • Ensure durable/idempotent execution, retries and dead-letter handling, and approvals for irreversible actions
• Create an evaluation framework
  • Build golden datasets from observed production failures
  • Define retrieval and system metrics
• Improve observability across the full request path
• Optimize cost and latency
  • Prompt caching, batching, model routing, and using smaller models where appropriate
• Use deterministic logic where it’s better
  • Identify opportunities and implement accordingly
• Collaborate with non-technical colleagues to specify and validate automated processes

Requirements

• Production experience with at least one LLM-based system, including operational ownership and incident handling
• Advanced TypeScript and Node.js
  • Strict typing of nondeterministic model output
  • Async/concurrency patterns, streaming responses, structured error handling
• Next.js in production
  • App Router, route handlers, server actions, streaming to the client
• Proven experience taking over and improving existing code under production traffic
• Practical retrieval expertise
  • Hybrid search, embedding selection, cross-encoder reranking
  • Metadata filtering, permission-aware retrieval, diagnosis of poor retrieval quality
• Document processing experience
  • PDFs (including tables), scanned material, DOCX, HTML
  • Layout-aware parsing, OCR, evidence-based chunking
• Structured outputs and tool calling
  • JSON Schema / Zod (or similar) runtime validation
  • Function calling; handling malformed/partial output
  • Context window management
• Designed and run LLM evaluations
• Experience with LLM tracing/evaluation tooling in a TypeScript codebase (e.g. Braintrust, Langfuse, Promptfoo, OpenTelemetry, Arize Phoenix)
• Familiar with Postgres + vector search (pgvector or similar), Docker, Git, CI/CD, and one major cloud platform
• Working experience with Anthropic and/or OpenAI TypeScript SDKs
• Confident communication in English; German is a plus

Nice to have

• Durable workflow execution for long-running unattended processes (e.g. Temporal, Inngest)
• Agent orchestration in production and multi-step workflows (e.g. Vercel AI SDK, LangGraph, Mastra, Claude Agent SDK, MCP TypeScript SDK)
• Integration experience with enterprise systems such as ERP