Data Platform Engineer (26-27)

Role Overview

The Data Platform Engineer builds and operates IDEA’s data infrastructure on Snowflake. You will design automated ingestion pipelines, optimize for performance and cost, and ensure the platform runs as a production-grade system for downstream analytics and research teams.

Reporting to the Manager of Data Platform Engineering, this is a hands-on individual contributor role (no direct reports) supporting IDEA’s move from legacy ETL to a modern lakehouse architecture.

Responsibilities

• Design, build, and maintain automated ELT pipelines to ingest data from diverse source systems into Snowflake
• Configure cloud-native ingestion tools and create custom Python-based pipelines when needed
• Build and maintain Bronze-layer tables with schema evolution, audit metadata, and lineage
• Implement ingestion-level validation and monitoring to detect issues early
• Document source configurations, refresh schedules, and troubleshooting procedures
• Partner with Analytics Engineering to ensure ingestion supports downstream transformations

Snowflake Administration & Platform Operations

• Administer Snowflake: databases, schemas, warehouses, access controls, and security settings
• Manage Snowflake objects using infrastructure-as-code patterns
• Implement security best practices: RBAC, encryption, auditing, and network policies
• Evaluate and adopt new Snowflake capabilities as appropriate
• Own Terraform-based infrastructure definitions for Snowflake and related components
• Automate recurring operational tasks (provisioning, access grants, environment setup)
• Build CI/CD pipelines for infrastructure changes with safe deployment practices
• Maintain reusable templates/modules for onboarding new sources and domains
• Create and maintain documentation and runbooks for platform operations
• Implement monitoring and alerting for pipeline health, platform health, and performance
• Troubleshoot failures using systematic root-cause analysis
• Embed observability (logging, metrics, alerts) into production pipelines
• Participate in code reviews, design discussions, and retrospectives; mentor peers and share platform knowledge

Location & Travel

• Full-time remote role based in Texas
• Preference for candidates in: Austin, El Paso, Houston, Permian Basin (Midland/Odessa), Rio Grande Valley, San Antonio, Tarrant County (Fort Worth), or those willing to relocate
• Minimal travel (5–10% annually)

Requirements / Skills (from posting)

• Strong ability to build production data ingestion and platform operations on Snowflake
• Hands-on experience with ELT pipelines, data validation/monitoring, and observability
• Experience with infrastructure-as-code and Terraform
• Competence with CI/CD for infrastructure changes
• Knowledge of security practices: RBAC, encryption, auditing, network policies

Nice-to-have / Additional signals

• Experience transitioning from legacy ETL to a lakehouse architecture
• Ability to mentor peers and lead technical initiatives as needed