Big Data Engineer

Role Overview

Design and deliver scalable, secure data pipelines and models that connect operational systems to analytics. Build batch, CDC, and streaming data flows from diverse sources and support certified Power BI datasets and APIs.

Responsibilities

• Develop and maintain batch, CDC, and streaming pipelines from multiple data sources
• Create dimensional and semantic models to power certified Power BI datasets and APIs
• Collaborate on OT/IT integration and ensure data quality, security, and observability
• Automate build, test, and deployment using Git-based CI/CD
• Partner with BI teams on semantic model contracts, RLS, and performance SLAs

Requirements

• 5+ years building production data pipelines (batch/CDC/streaming)
• Hands-on experience with the Azure data stack:
  • Databricks/Fabric/Synapse, ADF, ADLS, Azure SQL, Key Vault
• Strong SQL skills and Python/PySpark experience, including:
  • Spark Structured Streaming
  • performance tuning
• Experience with testing, observability, and Git-based CI/CD

Nice to Have

• Familiarity with SAP S/4HANA structures and SAP DataSphere semantic modeling
• Understanding of OT concepts such as historians, OPC UA/MQTT, and ISA-95/99 basics

What’s Offered

• Work on an enterprise Unified Data Layer spanning IT and OT
• Support critical functions such as Finance, Operations, and Supply Chain
• Exposure to advanced practices in data quality, security, and observability