Opportunity Description
Key Responsibilities
- Pipeline Development: Design, develop, and maintain robust end-to-end data pipelines and orchestration workflows using Python , Apache Airflow , and Cloud Composer on GCP.
- Data Modernization: Bridge the gap between legacy/on-premise database environments and modern cloud architectures, ensuring seamless data ingestion and migration.
- Optimization & Performance: Architect and tune analytics solutions at scale within BigQuery , utilizing partitioning, clustering, and materialized views to maximize efficiency and control costs.
- CI/CD & Software Best Practices: Embed rigorous software engineering principles, including Git/GitHub version control, strict PR disciplines, automated unit/integration testing, and deployment via CI/CD pipelines.
- Data Governance & Compliance: Embed data quality controls, access management, data lineage, and privacy frameworks ...