Opportunity Description
Job Description
Join our exciting team as a Data Engineer with the following job scope:
- Architect and implement data solutions on our large‑scale data platform Hadoop using PySpark.
- Write complex and efficient SQL queries to extract and transform data from various sources.
- Design, build, and maintain scalable and reliable data pipelines using Apache Airflow and Kafka.
- Develop and optimize data processing jobs on the Microsoft Azure cloud platform, leveraging Azure Data Lake, Databricks, and Azure Data Factory.
- Collaborate with data analysts and other stakeholders to understand data requirements and deliver high‑quality data products.
- Ensure data quality and integrity across all systems.
- Monitor, troubleshoot, and optimize the performance of our data infrastructure.
- Stay up‑to‑date with the latest trends and technologies in the Big Data space.