Responsible AI Governance & AI Technology Assessment

Problem

AI use cases were spreading across the holding company and its subsidiaries with no shared governance standard, so each team chose tools and managed risk differently.

Approach

Working with client stakeholders and teams across countries, we paired a technology gap assessment across the AI lifecycle with deep risk assessments of selected use cases. In the risk work, qualitative screening across eight Responsible AI dimensions came first, followed by quantitative tests on the riskiest dimensions against agreed thresholds.

How It Works

1. 1
   DiscoverQuestionnaires and workshops with use-case owners across business units, compiled into a current-state view.
2. 2
   Gap analysisEach use case compared with best practice across ideation, data, model development, and deployment, scored on people, process, and technology.
3. 3
   RecommendEach gap mapped to a priority and candidate tooling; a selection guide covers reference architecture, build-versus-buy, RAG and agent patterns, LLM benchmarking, RAG versus fine-tuning, and cost.
4. 4
   Risk screeningUse cases rated on eight dimensions: compliance, fairness, transparency, accountability, sustainability, privacy, soundness, and robustness.
5. 5
   Quantitative testsWith the testing toolkit: SHAP explainability on an anomaly-detection model, and answer-correctness, toxicity, bias, and PII tests on a RAG chatbot (Ragas, DeepEval, Detoxify, Presidio, LLM Guard).
6. 6
   MitigateFindings mapped to a risk and control taxonomy, with mitigation plans and a 12-month governance roadmap.

What I Built

• A Responsible AI testing toolkit (Streamlit, Ragas, Presidio, Detoxify, LLM Guard) for assessing ML and RAG use cases on toxicity, PII, bias, explainability (SHAP), and answer correctness.

Impact

Gave the client a common baseline for choosing AI technology and a repeatable way to test AI systems before release. The tests also flagged a chatbot whose robustness score fell below the agreed threshold.