Senior Data Engineer (Pyspark)

Role & responsibilities

• The Data Engineer role involves acquiring requirements, conducting EDA, ingesting required datasets, and transforming data using Big-data technologies & Feature Engineering techniques for machine learning models.
• Key tasks include building high-performance, secure, and scalable data pipelines, collaborating with Analytics Delivery Leads, Data scientists and other teams in an Agile environment, and communicating stakeholders effectively.
• The position requires a degree in a relevant field.
• At least 8-10 years of experience in data engineering and ML pipelines, and proficiency in technologies like Python, Spark (including expertise in optimization techniques), Hadoop, SQL, and Git.