Data Engineer I

Role Overview

As a Data Engineer I at Vālenz Health®, you will build and support scalable data pipelines in a cloud-based lakehouse environment. You’ll help ingest healthcare data, validate and enrich it, and deliver reliable datasets for analytics and reporting across the organization.

Responsibilities

• Build and maintain processes to acquire, validate, and enrich data from multiple sources.
• Support migration of on-premise SQL Server data to a cloud lakehouse architecture using Azure Databricks and Delta Lake, including transformations and pipeline re-architecture.
• Develop and optimize ETL/ELT pipelines using PySpark and Spark SQL.
• Implement Lakehouse + Delta best practices, including schema enforcement, ACID transactions, and data versioning.
• Orchestrate pipelines using Databricks Workflows (Jobs) or similar tools.
• Implement data quality frameworks, validation checks, and monitoring for pipeline reliability.
• Optimize pipeline performance and cost.
• Collaborate on CI/CD for data pipelines (testing, deployment, versioning).
• Partner with analytics and business stakeholders to identify new data sources and assess feasibility.
• Design data models for analytics, reporting, and data warehousing use cases.
• Participate actively in agile processes.

Requirements

• 1+ years of work experience in a data engineering role.
• Bachelor’s degree (or greater) in a quantitative field (or equivalent practical experience).
• Hands-on experience with Databricks (Spark, PySpark) and Delta Lake, and/or migrating RDBMS systems to a lakehouse.
• Experience with healthcare data types (e.g., medical claims, eligibility, provider network rosters, Rx claims).
• Strong organization, time management, and ability to build/re-evaluate processes from the ground up.
• Strong investigative skills and high attention to detail (including comfortable work with messy/ambiguous data).
• Hands-on SQL and Python (PySpark) for distributed processing.
• Experience building and optimizing large-scale distributed pipelines for batch and streaming ingestion.

Nice-to-Haves

• Comfortable collaborating across analysts, data scientists, and stakeholders to expand data architecture as new sources emerge.