Data Engineer
Haystack
midpermanentdatabackend United States Today via LinkedIn
See how well this job matches your profile
Sign up to get an AI match score and generate a tailored application in seconds.
Get your match scoreTags
DatabricksApache SparkPySparkDelta LakeLakehouseETL/ELTSQLPythonAzure DatabricksAWS Databricks
About the role
Role Overview
Design, develop, and maintain scalable lakehouse data pipelines using Databricks and optimize data platform performance for enterprise use cases.
Responsibilities
- Build and maintain scalable ETL/ELT pipelines using Databricks
- Design and optimize Delta Lake / Lakehouse architectures
- Develop data ingestion frameworks for structured and unstructured data
- Implement data governance, security, and performance optimization strategies
- Collaborate with data architects, analysts, and business stakeholders
- Optimize Spark workloads for scalability and cost efficiency
Requirements
- Databricks Partner Certification (mandatory)
- Strong expertise in Databricks Lakehouse Platform and Apache Spark (PySpark/Scala)
- Proficiency with Delta Lake, Python, and SQL
- Experience with Azure Databricks or AWS Databricks
- Solid understanding of data engineering, ETL development, and performance tuning
Nice to Have
- Familiarity with data modeling
- Experience with Git, CI/CD, and cloud storage (ADLS, S3)
About Haystack
Haystack works with companies focused on data platform innovation, delivering scalable and optimized data solutions for enterprises. The posting emphasizes modern lakehouse technologies and enterprise-grade data engineering practices.
Scraped 8/6/2026