AI
Why AI Systems Need Governance Differently
Traditional software runs on explicit rules, predictable and traceable. AI systems learn patterns from data and change over time. Behavior isn’t fixed. A loan AI decides approvals in milliseconds without human review. If it denies a qualified applicant, who is accountable? Failure is subtle: drift, bias, unfair outcomes without alarms. Governance designed for static code cannot supervise probabilistic systems.
Inventory Systems and Score Materiality
First, know what AI systems exist. Build a service catalog. Then determine which need oversight. Not all require governance — a chatbot moving data from A to B has low autonomy. Three factors determine materiality: impact on customer or business outcomes, autonomy (independent decision-making), and sensitivity (PII, HIPAA data). On a 1-3 scale, the loan AI scores 3 on all three — impact 3, autonomy 3, sensitivity 3. Summation gives 9 out of 9; it needs governance.
Ownership, Explainability, and Auditability
Accountability requires answering three questions: what it did, why it did it, who owns it, and can you prove it. Ownership is redistributed: model owner for integrity, business owner for ethical outcomes, GRC for oversight. Explainability answers what and why — without it, the system is a black box. Auditability provides structured evidence over time: logs of outcomes, retraining data, decision paths. Auditors must reproduce the same decision given same inputs. These three form the minimum viable reliability model for AI.
Governance, Risk, and Assurance Controls
Controls are boundaries. Governance controls set roles and policies — like a steering wheel. For the loan AI: business owner sign-off before deployment, ethics review, annual recertification. Risk controls ensure fairness and stability — like brakes. Quarterly bias testing, fairness monitoring, explainability logs. Assurance controls audit performance — like a dashboard. Semi-annual internal or external audits verify reproducibility and fairness. Audit is continuous assurance, not a one-time check.
From Blind Adoption to Continuous Assurance
Three shifts: from blind adoption to risk-based oversight, from black box to accountable system, from static reviews to continuous assurance. AI doesn’t remove accountability — it redistributes it. No new team or framework needed. Inventory systems, assign owners, score materiality, add drift and explainability metrics, treat retraining like deploys. If a model decides who gets a loan, it deserves more governance than a human loan officer.
Q&A
Is this methodology your creation? No, it adapts existing frameworks for AI systems. ▶ 22:14
Doesn’t adding oversight defeat the purpose of AI by creating overhead? The practices already exist; just adapt existing risk matrices for AI. ▶ 23:49
How do you add explainability metrics from LLM output when chain-of-thought isn’t a metric? GRC controls monitor drift in decision paths and fairness criteria to ensure new criteria added by the model are ethical. ▶ 25:16
Notable Quotes
AI doesn’t remove accountability, like I said. It just redistributes it Pavithra Pradip · ▶ 20:42
AI systems rarely fail loudly, right? They’re always about subtle failure. It’s just they drift over time. There is bias, there is unfairness, and there is bad outcomes over time, but you don’t notice it Pavithra Pradip · ▶ 7:37
If a model can decide if someone gets a loan, I think it should probably get maybe the same or maybe a little bit more governance than a human loan officer Pavithra Pradip · ▶ 20:32
So just to recap, it’s not about AI doesn’t remove accountability, it’s just redistributing it Pavithra Pradip · ▶ 15:21
Key Takeaways
- Inventory AI systems and build a service catalog for governance.
- Score materiality with impact, autonomy, and sensitivity to prioritize oversight.
- Apply ownership, explainability, and auditability as the minimum viable reliability model.
About the Speaker
Pavithra Pradip is a Staff Technical Compliance Manager with extensive experience in business and information security. She specializes in governance within GRC, driving the development and enhancement of security policies and standards. Her ability to connect technical and business objectives…