Data Quality Engineer (Databricks)

Develop Delta Lake metric aggregation structures supporting enterprise quality reporting. Calculate and publish: Data Quality Index (DQI) scores; dimension-level quality metrics; rule pass/fail rates; dataset compliance scores; SLA adherence indicators. Provide curated outputs to support Power BI quality dashboards and executive reporting.

Configure automated quality monitoring and alerting mechanisms. Implement threshold-based notifications using Databricks SQL Alerts, Azure Monitor integrations. Develop predictive risk scoring models to identify datasets at risk of future quality degradation. Support proactive quality management and operational intervention activities.

Apply Databricks machine learning and pattern analysis techniques to profiling and rule execution outputs. Support AI-assisted root cause analysis across established remediat...