Staff Software Engineer (Data Governance & Foundations)
Instacart
full-remoteleadpermanentdataengineering-management Full remote - New-York, US 98 days ago via WTTJ
See how well this job matches your profile
Sign up to get an AI match score and generate a tailored application in seconds.
Get your match scoreTags
Data GovernanceData LakehouseApache IcebergDelta LakeHudiTrinoSparkClickHouseKafkaStreaming Infrastructure
About the role
Role Overview
Staff Software Engineer on the Data Governance & Foundations team at Instacart. You will be a senior technical leader responsible for the architecture and delivery of an open lakehouse foundation, including governance/access patterns and a multi-engine compute strategy.
Key Missions / Responsibilities
- Architect and deliver the open lakehouse foundation, governance & access patterns, and multi-engine compute strategy.
- Collaborate with engineering leadership and cross-department stakeholders to align Instacart’s data strategy with a multi-year architecture roadmap.
- Drive real-time & streaming infrastructure for critical use cases, including:
- deployment patterns
- SLAs
- operational practices
- Own the open lakehouse foundation and advance real-time and streaming infrastructure.
- Pioneer AI-native data infrastructure engineering.
- Elevate engineering excellence across the data platform organization.
Requirements
- 10+ years of software engineering experience building and operating data infrastructure or distributed systems at production scale.
- Hands-on expertise with modern lakehouse architectures and open table formats (e.g., Apache Iceberg, Delta Lake, Hudi).
- Hands-on experience with distributed query/compute engines (e.g., Trino, Spark, ClickHouse), including performance tuning and production reliability.
- Ability to build cost/benefit and TCO models for infrastructure investments.
- Experience designing platform-level governance controls and familiarity with compliance frameworks such as SOX, CPRA, GDPR.
- Experience with event-driven/streaming infrastructure (e.g., Kafka, Flink).
- Proven ownership of major platform transitions/migrations (build vs. buy, migration design, risk management) to production.
- Strong SQL skills and systems-level development skills in Python or Scala.
- Experience with orchestration and transformation pipelines at scale:
- Apache Airflow
- dbt
Nice-to-Haves (implied)
- FinOps experience optimizing data platform spend, including managing large infrastructure budgets and vendor negotiations.
- Advanced academic background (BS/MS/PhD in CS or equivalent) or equivalent practical experience.
About Instacart
Instacart is a technology company focused on building large-scale consumer and commerce experiences. It operates data-intensive platforms and engineering teams that support data infrastructure, governance, and analytics capabilities at production scale.
Scraped 8/16/2026