Data Engineer (Pilar)

Join to apply for the Data Engineer role at Kelsus, Inc.

• Design and develop efficient data pipelines using Python, AWS Glue/pyspark, AWS Lambda, Kafka, and related technologies.
• Implement and optimize ETL processes for data extraction, transformation, and loading into relational databases, especially Postgres.
• Collaborate on data warehouse architectures, ensuring proper data modeling, storage, and access.
• Utilize tools like StitchData and Apache Hudi for data integration and incremental management, improving efficiency and enabling complex operations.
• Identify and resolve data quality and consistency issues, implementing monitoring processes across pipelines and storage systems.

• Strong Python skills and proficiency in associated libraries for data processing and manipulation.
• Expertise in AWS Glue/pyspark, AWS Lambda, and Kafka for ETL workflow development and stre...