About the role

Role Overview

Build and operate an enterprise Unified Data Layer (UDL) spanning IT and OT systems. You will engineer batch, CDC, and streaming pipelines and model curated/semantic layers to support business domains and analytics.

Responsibilities

Build and operate the enterprise Unified Data Layer (UDL) across IT and OT.
Engineer batch, CDC, and streaming data pipelines to deliver data products.
Model curated and semantic layers for business domains.
Harden run-state using testing, CI/CD, security, and observability practices.
Partner with data teams and IT to deliver scalable data solutions.
Design and deliver secure data pipelines and models connecting operational systems to analytics.

Requirements

5+ years of data engineering experience building production pipelines at scale.
Hands-on experience with the Azure data stack:
- Databricks
- Fabric/Synapse
- ADF/Pipelines
- ADLS/OneLake
- Azure SQL / SQL MI
- Key Vault
Strong SQL and Python/PySpark skills.
Expertise in Spark Structured Streaming and performance tuning.
Experience implementing data quality rules and unit/integration testing.
Git-based CI/CD experience.

Nice-to-Haves

Familiarity with SAP S/4HANA structures and SAP DataSphere semantic modeling.
Understanding of OT concepts:
- Historians (PHD/PI)
- OPC UA / MQTT
- Event/batch frames
- ISA-95/99 basics.

What’s On Offer

Work on innovative data solutions supporting critical business functions.
Collaborative environment focused on cutting-edge data technologies.
Continuous learning and professional development.

About Haystack

Haystack is partnering with an innovative company focused on data management and analytics across industries such as finance and supply chain. They deliver robust, trustworthy, and high-performance data products, including unified data layer capabilities and advanced analytics foundations.

Data Engineer

Tags