AVP/VP, AI/ML Model Validation Engineer, Data Management Office

Responsibilities

• Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models.

• Perform functional, regression and scenario‑based testing of model behaviours and workflows.

• Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection.

• Assess end‑to‑end model workflows including data inputs, feature transformations, task completion, tool‑use accuracy and multi‑step reasoning.

• Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks.

• Validate API and tool‑integration behaviour in production‑like environments, identifying dependency or orchestration issues.

• Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly.

• Collaborate with data scientists across departments to understand modelling intent, f...