Data Engineer

<p>The overall technical lead and architect. Designs the metadata schema, builds the simulation onboarding pipeline, deploys metadata embedding pipeline and OpenSearch k-NN vector store, and authors data export format spec for AI/ML use case. This role is the deepest technical seat on the engagement:</p> <p> </p> <p><b><u>Key responsibilities on this engagement</u></b></p> <p> Run the Sprint 1 architecture review of the existing UAT codebase (S3 + Glue + S3 Tables + OpenSearch + Athena) and deliver written gap findings.</p> <p> Design the metadata schema, taxonomy, and field catalogue (Light, Brain, Power).</p> <p> Tune data orchestration - Glue jobs, Athena queries, S3 Tables config, scheduling. Lead the deep-dive technical sessions with analysts on visualization requirements </p> <p> Build and validate the simulation data onboarding pipeline against real data - includin...