Data Engineer (Lakehouse & Hybrid Data Platforms)
My client is looking for a Data Engineer to design and build scalable data pipelines that deliver trusted, analytics‑ready datasets for BI, AI, and operational use cases across a hybrid environment.
Key Responsibilities
- Build pipelines across bronze, silver & gold layers (Databricks, Spark, dbt)
- Implement data quality checks, contracts & schema validation
- Apply governance (catalog, lineage, RBAC, metadata)
- Deliver curated datasets, features & embeddings for AI/BI
- Monitor pipeline health, performance & cost to meet SLAs
Tech Stack
- Databricks, Spark, Delta Lake, dbt
- Azure Data Factory, Kafka/Event Hubs
- CI/CD (Azure DevOps/GitHub)
Governance & Ops
- Enforce data contracts, lineage & cataloging
- Apply masking, tokenisation & access controls (PII/PHI)
- Build observable pipelines with alerts, dashboards & runbooks
- Optimize performance (partitioning, caching, cost efficiency)
Requirements
- 5+ years in Data Engineering
- Strong SQL, data modeling (dimensional/data vault)
- Proficiency in Python
- Hands‑on with Databricks, Spark, Delta Lake & dbt
- Experience with Azure data services (ADF, ADLS, Key Vault)
- Familiarity with CI/CD & container basics (Docker/Kubernetes)
Nice to Have
- Streaming (Kafka/Event Hubs) & CDC (GoldenGate)
- Catalog/lineage tools (Purview, OvalEdge)
- S3‑compatible storage (MinIO, VAST)
- Exposure to BI tools (Power BI) & healthcare standards (FHIR/MDR)
Education
- Bachelor's in Computer Science, Engineering, or related field
Salt is acting as an Employment Agency in relation to this vacancy.
#J-18808-Ljbffr
This listing is sourced from a third-party job board. Applying will redirect you to the original posting.