Description
Job title: LLM Research & Validation Specialist
Location: Abu Dhabi UAE
Role purpose:
Lead frontier research quantitative evaluation and independent validation of large language models multimodal models retrieval-augmented generation systems and agentic AI used or proposed by ADIB. Translate mathematical and scientific methods into reproducible validation tests challenger analyses runtime controls and decision-useful evidence for model governance.
The role combines deep technical research with second-line effective challenge.
It is expected to build validation toolkits and evaluation harnesses independently assess conceptual soundness and production behaviour and communicate material limitations clearly to technical teams senior management and governance forums.
The role does not own model development or production approval.
Key accountabilities /responsibilities:
Lead independent validation of LLM multimodal RAG and agentic AI use cases across design implementation deployment and ongoing monitoring.
Assess transformer architecture tokenisation embeddings attention context-window behaviour decoding fine-tuning alignment quantisation and inference configuration.
Design reproducible evaluation harnesses golden datasets adversarial suites counterfactual tests canary sets and statistically defensible acceptance criteria.
Evaluate task performance hallucination and factuality calibration robustness stability long-context behaviour retrieval quality grounding citation faithfulness and uncertainty.
Perform deep testing of prompt injection indirect injection data leakage tool-use safety excessive agency multi-step failure propagation kill-switches and human oversight.
Apply probability statistics optimisation information theory numerical methods and experimental design to develop challenger tests and quantify uncertainty.
Review data provenance representativeness contamination benchmark validity leakage drift and limitations of synthetic or LLM-generated evaluation data.
Build and maintain reusable Python-based validation tooling automated test pipelines experiment tracking results repositories and technical documentation.
Conduct structured research on emerging model architectures interpretability mechanistic analysis scalable oversight model evaluation and AI safety methods.
Independently challenge model owners vendors and developers document findings propose risk-based restrictions and track remediation without assuming first-line ownership.
Prepare validation reports research notes standards committee papers and senior-management briefings that clearly distinguish evidence judgement and residual uncertainty.
Mentor junior validators improve team methodology and support knowledge transfer across Model Risk
Education and experience:
Masters degree in Theoretical Physics Applied Physics Mathematics Applied Mathematics or a closely related quantitative discipline is required. A PhD or research-intensive masters is strongly preferred.
Typically one to three years of relevant experience in AI research machine learning quantitative modelling model validation scientific computing or a closely related field. Exceptional research profiles may be considered based on demonstrated capability.
Deep understanding of probability statistics linear algebra optimisation numerical computation experimental design and uncertainty quantification.
Strong understanding of transformers LLM training and inference embeddings RAG fine-tuning alignment evaluation agentic systems and AI safety failure modes.
Advanced Python proficiency and experience with scientific and ML libraries. Exposure to PyTorch Hugging Face evaluation frameworks experiment tracking SQL Git and cloud AI platforms is expected.
Ability to read research papers critically reproduce methods design-controlled experiments and convert findings into bank-grade validation evidence.
Experience with red teaming adversarial testing interpretability calibration robustness privacy security or model risk management is strongly advantageous.
Excellent technical writing and communication including the ability to explain mathematical concepts assumptions and limitations to non-specialist stakeholders.
Banking experience is advantageous but not mandatory. The role requires willingness to develop knowledge of financial services Islamic banking CBUAE expectations and ADIB governance.
Indicative success measures:
Validation conclusions are reproducible evidence-based and proportionate to use-case risk.
Reusable evaluation assets and automation measurably improve validation coverage consistency and efficiency.
Material LLM and agentic risks are identified early clearly communicated and translated into actionable controls or use restrictions.
Research outputs strengthen ADIB validation methodology and remain traceable to tested evidence rather than unsupported claims.
Stakeholders receive constructive independent challenges while second-line ownership and decision rights remain clear.
Required Experience:
IC
This listing is sourced from a third-party job board. Applying will redirect you to the original posting.
Minor International
YOSH
Marriott Hotels Resorts
Elwood Roberts