Opportunity Description
Location: Indore/RaipurExperience: 2 to 4 yearsKey Responsibilities:
+ Design, develop, and manage scalable data pipelines and ETL workflows using Databricks, PySpark, and SQL for large-scale data processing.
+ Build and maintain data ingestion frameworks to extract data from enterprise systems such as SAP APIs, REST services, and relational databases.
+ Develop and optimize Delta Lake based data architecture to ensure reliable, high-performance data storage and processing.
+ Design and implement data transformation pipelines to convert raw data into curated datasets for analytics and reporting.
+ Optimize Spark jobs and SQL queries to improve performance and reduce compute costs.
+ Implement data quality validation, monitoring, and error handling frameworks for reliable pipeline execution.
+ Build automated workflow orchestration and scheduling mechanisms for end-to-end data processing pipelines.
+ Collaborate wi...
+ Design, develop, and manage scalable data pipelines and ETL workflows using Databricks, PySpark, and SQL for large-scale data processing.
+ Build and maintain data ingestion frameworks to extract data from enterprise systems such as SAP APIs, REST services, and relational databases.
+ Develop and optimize Delta Lake based data architecture to ensure reliable, high-performance data storage and processing.
+ Design and implement data transformation pipelines to convert raw data into curated datasets for analytics and reporting.
+ Optimize Spark jobs and SQL queries to improve performance and reduce compute costs.
+ Implement data quality validation, monitoring, and error handling frameworks for reliable pipeline execution.
+ Build automated workflow orchestration and scheduling mechanisms for end-to-end data processing pipelines.
+ Collaborate wi...