Data Engineer (Real time Streaming)

JD -

- 5+ years hands-on Azure Databricks experience.
- Build real-time ingestion pipelines with Azure Event Hub and Databricks Structured Streaming.
- Design scalable streaming pipelines following best practices.
- Familiarity with Unity Catalog and data governance principles.
- Implement structured logging and metadata augmentation.
- Unit testing and DevOps practices for data workloads.
- Proficient in SQL and Python programming and advanced software development and DevOps concepts.
- Implement Azure Functions for XML/XSD validation.
- Optimize Spark performance in Databricks.
- Proficient in Python XML/XSD parsing/validation and modular packages.
- Hands-on with Azure DevOps/GitHub.
- Set up monitoring/alerting via Azure Monitor, Databricks Jobs, Log Analytics.
- Basic Terraform/IaC knowledge.
- Collaborate cross-functionally; adopt new tech; apply data engineering principles.
- Databricks Certified Associate/Professional Data Engi...