Senior Software Engineer (Applied Artificial Intelligence)

About the role

Join Smartsheet, a leading platform for work management and automation. As a Senior Software Engineer, you will be part of the AI Platform Engineering team, responsible for building the core infrastructure for AI experiences, standardizing AI development, and ensuring trust and safety for AI-driven features. This full-time position offers remote work options and a comprehensive benefits package. Key missions: Lead the design and ownership of the core infrastructure that serves as the backbone for all Smartsheet AI experiences.. Architect high-level abstractions and "Golden Path" APIs that democratize AI development across Smartsheet.. Establish the mission-critical monitoring and quality assurance layers that protect Smartsheet customers. Profile: - Ability to communicate complex quality findings (written and verbal) to both technical and non-technical stakeholders, you can explain what’s broke, why it matters, and what needs to happen next without losing the room - Deep, hands-on experience with prompt engineering and context engineering, you understand how model behavior changes with framing, structure, and input design - Strong working knowledge of RAG architectures: chunking strategies, embedding models, retrieval evaluation, and failure diagnosis - 8+ years of software engineering experience, with at least 2 years working directly with LLMs in production - A bias for clarity in ambiguous situations, when failure modes are murky and trade-offs are real, you bring structure and a clear point of view rather than waiting for consensus - Experience building or extending LLM evaluation frameworks, you have designed scorers, worked with golden datasets, and thought carefully about what good looks like - Strong Python skills; comfortable working in data-heavy environments (Databricks, Delta tables, or equivalent) - Strong cross-functional judgment, you know when to escalate, when to resolve independently, and how to build credibility across engineering, product, and AI platform teams - Prior work in an Applied AI or LLMOps platform within a product company - Kubernetes (EKS/GKE): The industry standard for AI. Skills include managing GPU scheduling, auto-scaling based on token throughput, and using tools like Karpenter for cost-efficient node provisioning - Infrastructure as Code (IaC): Using Terraform, Pulumi, or AWS CDK to provision Vector Databases, SQS queues, and S3 buckets - Vector Databases: Proficiency in managing and optimizing Pinecone, Milvus, Weaviate, or Databricks Vector Search - AI Gateways: Building or configuring proxies (like LiteLLM or Kong AI Gateway) to handle rate-limiting, PII masking, and cost-tracking - Model-Based Evals: Implementing automated scoring systems (like RAGAS or DeepEval) that use an "LLM-as-a-Judge" to grade production outputs - LLM Observability: Setting up tracing tools like Langfuse, LangSmith, or MLflow to monitor "Time to First Token" (TTFT) and trace hallucination issues