Databricks Solutions Architect

Role Overview

540 is seeking a Databricks Solutions Architect to act as an embedded technical leader for customer engagements under the Databricks Partner Program (DPP). You will provide hands-on architectural leadership for complex Databricks implementations, serving as a long-term extension of Databricks Professional Services.

Responsibilities

• Serve as the embedded technical lead for Databricks customer engagements
• Own Databricks platform architecture, design decisions, and technical standards
• Lead delivery of complex data pipelines and analytics workloads on Databricks
• Provide deep expertise in Apache Spark internals, optimization, and scalability
• Guide customers on Databricks best practices across ingestion, transformation, and analytics
• Partner with Databricks Professional Services leadership on delivery strategy
• Mentor and oversee partner and customer delivery teams
• Ensure solutions meet Databricks PS governance, security, and quality standards
• Drive adoption, platform maturity, and long-term customer success

Requirements

• 10+ years in consulting or related client-facing roles
• 7+ years in data engineering or related technical roles
• Databricks Data Engineering Professional certification
• Hands-on architectural leadership on 6+ Databricks projects
• Deep expertise in Apache Spark (runtime internals and optimization)
• Strong experience designing large-scale distributed data platforms
• Advanced SQL and Python expertise (Scala strongly preferred)
• Strong working knowledge of at least one major cloud (AWS, Azure, or GCP)
• Experience in regulated or enterprise-scale environments
• U.S. Citizenship and ability to obtain a DoD Secret clearance (per client requirements)
• Bachelor’s degree in Computer Science or related engineering field (preferred)

Nice to Have

• Prior experience serving as a Solutions Architect or long-term embedded consultant
• Expertise across multiple clouds (AWS/Azure/GCP)
• Familiarity with CI/CD, infrastructure-as-code, and production deployment patterns
• Working knowledge of MLOps and advanced analytics workloads
• Experience with Unity Catalog, data governance, and security architectures
• Experience enabling/mentoring consulting teams at scale
• Exposure to Databricks enablement programs (Partner Academy, Train-the-Trainer models)

Location / Work Model

• Remote within the continental United States
• Collaboration hours aligned with Eastern Time with flexibility
• Occasional travel