Data Platform Engineer (26-27)

Role Overview

The Data Platform Engineer builds and operates IDEA’s data infrastructure on Snowflake. You will design automated ingestion pipelines, optimize platform performance and cost, and ensure the platform runs as a production-grade system supporting analytics, reporting, and research across multiple states.

This is a hands-on role reporting to the Manager of Data Platform Engineering, contributing to IDEA’s shift from legacy ETL toward a modern lakehouse architecture.

Location & Logistics

• Full-time remote role based in Texas
• Preference for candidates in: Austin, El Paso, Houston, Permian Basin (Midland/Odessa), Rio Grande Valley, San Antonio, Tarrant County (Fort Worth)
• Minimal travel (5–10%) for collaboration, training, or critical milestones
• No direct reports (individual contributor)

Responsibilities

• Design, build, and maintain automated ELT pipelines to ingest diverse source systems into Snowflake
• Configure and manage cloud-native ingestion tools and custom Python-based pipelines when needed
• Build and maintain Bronze-layer tables with:
  • schema evolution
  • audit metadata
  • lineage
• Implement ingestion-level validation and monitoring to detect issues early
• Document source configurations, refresh schedules, and troubleshooting procedures
• Partner with Analytics Engineering so ingestion patterns support downstream transformation needs

Snowflake Administration & Platform Operations

• Administer Snowflake environments: databases, schemas, warehouses, access controls, and security settings
• Optimize performance and cost via warehouse sizing, clustering, query analysis, and resource monitoring
• Manage Snowflake objects using Infrastructure as Code (IaC) patterns
• Implement security best practices: RBAC, encryption, auditing, and network policies
• Evaluate and adopt new Snowflake capabilities as appropriate
• Own Terraform-based infrastructure definitions for Snowflake and related components
• Automate recurring operational tasks (provisioning, access grants, environment setup)
• Build CI/CD pipelines for infrastructure changes with safe deployment practices
• Create reusable templates/modules to accelerate onboarding of new sources/domains
• Maintain runbooks and operational documentation
• Implement monitoring and alerting for pipeline health and performance
• Troubleshoot pipeline/platform failures using root-cause analysis
• Embed observability (logging, metrics, alerts) into production pipelines
• Participate in code reviews and design discussions; mentor peers and contribute to retrospectives

Required / Expected Knowledge & Skills

• Strong decision-making based on data, feedback, and insights
• Ability to set clear goals, manage work effectively, and meet deadlines

Nice-to-Haves

• Ability/willingness to mentor peers and lead technical initiatives (for senior engineers)
• Familiarity with transitioning from legacy ETL to lakehouse architecture