Senior Data Engineer

Role Overview

Join Socure’s Data Automation team as a Senior Data Engineer. You will design and build scalable data platforms and pipelines that support identity verification products and analytics, owning end-to-end delivery of complex data initiatives.

Key Missions

• Design and build batch and streaming data pipelines for automated ingestion, ML feature engineering, and analytics across multiple product domains.
• Own end-to-end delivery of complex/ambiguous data initiatives, including:
  • architecture, implementation, testing, deployment, monitoring, and documentation
• Partner with Data Science, Product, and Engineering teams to translate requirements into technical solutions.
• Develop and evolve the data platform, automate data operations, and optimize cost and performance.

Requirements

• Deep experience with distributed data processing, especially Apache Spark, including performance tuning and optimization.
• 5+ years of hands-on data engineering experience building and maintaining production-grade data platforms and pipelines.
• Strong understanding of data modeling and data warehousing, including partitioning and schema design for large datasets.
• Experience operating production pipelines: monitoring, alerting, incident response, and reliability improvements.
• Strong programming skills (e.g., Python or Scala) for data processing and SQL for analytics.
• Experience with AWS services for data solutions (e.g., EMR, Lambda, S3).
• Hands-on experience with data orchestration tools (e.g., Airflow, Step Functions).
• Solid software engineering practices: version control, CI/CD, testing strategies, and code review.
• Mentoring experience and contributions to engineering-wide standards/best practices.
• Familiarity with modern data platform patterns such as Data Lakehouse and Data Mesh.

Nice-to-Haves / Additional Signals

• Experience with streaming or near-real-time processing (e.g., Kafka, Kinesis).
• Experience with prompt engineering and GenAI/LLMs.

Soft Skills

• Strong communication and collaboration skills; ability to work effectively with both technical and non-technical stakeholders.