xelys jobs xelys jobs

Staff Software Engineer (Data Governance & Foundations)

Instacart

full-remoteleadpermanentdataengineering-management Full remote - New-York, US 98 days ago via WTTJ

See how well this job matches your profile

Sign up to get an AI match score and generate a tailored application in seconds.

Get your match score

Tags

Data GovernanceData LakehouseApache IcebergDelta LakeHudiTrinoSparkClickHouseKafkaStreaming Infrastructure

About the role

Role Overview

Staff Software Engineer on the Data Governance & Foundations team at Instacart. You will be a senior technical leader responsible for the architecture and delivery of an open lakehouse foundation, including governance/access patterns and a multi-engine compute strategy.

Key Missions / Responsibilities

  • Architect and deliver the open lakehouse foundation, governance & access patterns, and multi-engine compute strategy.
  • Collaborate with engineering leadership and cross-department stakeholders to align Instacart’s data strategy with a multi-year architecture roadmap.
  • Drive real-time & streaming infrastructure for critical use cases, including:
    • deployment patterns
    • SLAs
    • operational practices
  • Own the open lakehouse foundation and advance real-time and streaming infrastructure.
  • Pioneer AI-native data infrastructure engineering.
  • Elevate engineering excellence across the data platform organization.

Requirements

  • 10+ years of software engineering experience building and operating data infrastructure or distributed systems at production scale.
  • Hands-on expertise with modern lakehouse architectures and open table formats (e.g., Apache Iceberg, Delta Lake, Hudi).
  • Hands-on experience with distributed query/compute engines (e.g., Trino, Spark, ClickHouse), including performance tuning and production reliability.
  • Ability to build cost/benefit and TCO models for infrastructure investments.
  • Experience designing platform-level governance controls and familiarity with compliance frameworks such as SOX, CPRA, GDPR.
  • Experience with event-driven/streaming infrastructure (e.g., Kafka, Flink).
  • Proven ownership of major platform transitions/migrations (build vs. buy, migration design, risk management) to production.
  • Strong SQL skills and systems-level development skills in Python or Scala.
  • Experience with orchestration and transformation pipelines at scale:
    • Apache Airflow
    • dbt

Nice-to-Haves (implied)

  • FinOps experience optimizing data platform spend, including managing large infrastructure budgets and vendor negotiations.
  • Advanced academic background (BS/MS/PhD in CS or equivalent) or equivalent practical experience.

About Instacart

Instacart is a technology company focused on building large-scale consumer and commerce experiences. It operates data-intensive platforms and engineering teams that support data infrastructure, governance, and analytics capabilities at production scale.

Scraped 8/16/2026