RR Skillverse · Free Learning Handbook
RR Skillverse Module 6 of 12 Theory Handbook Finance + AI + Governance

Module 6 — Explainable & Responsible AI

Same theory-first structure as Modules 1–5: for every technique below, what it is, why it exists — the specific problem it was invented to solve — what else you could use instead, and a real financial example. Every module so far has answered "does it work?" and "can it be attacked?" Module 6 answers the question RR Finance has deferred since Module 1: "can we explain why it made this decision, and is that decision fair?"

⬇ Download the Module 6 notebook ← Back to Module 5 All modules →

Recap — what RR Finance already has

Module 6 is not a fresh start. Here's exactly what it inherits before turning the lens back on all of it.

MODULE 1

A validated logistic-regression baseline (test ROC-AUC ≈ 0.69) — and the age column, deliberately excluded from every model's features, flagged "for Module 6" since Lesson 1.

MODULE 2

A deeper ANN classifier, benchmarked against the Module 1 baseline — and adversarial attacks showing even a "working" model can be manipulated.

MODULE 3–5

Text understanding, an aligned language model built from scratch, and RAG-powered tool-using agents — including a rule-based routing decision this module explicitly returns to.

What's new in Module 6
Every prior module optimised for "does it work" and stress-tested "can it be attacked." Module 6 adds the question a real lender is legally required to answer for a real rejected applicant: why — and whether that "why" holds up the same way across every group of applicants.

Why explainability, why now

A model that cannot explain itself is not deployable in lending, no matter how accurate it is.

The legal reality

Under fair-lending frameworks like the US Equal Credit Opportunity Act, a rejected applicant is entitled to the specific reasons for that rejection — "the model said no" is not a compliant answer.

The technical gap

Module 1's logistic regression and Module 2's ANN both predict well enough to be useful and both remain, by default, black boxes about why any single prediction came out the way it did.

The deferred debt

age has sat in the dataframe since Module 1, excluded from every model's inputs but never actually audited for disparate impact — until this module.

CONCEPT 1

The Adverse-Action Problem

Why a rejected applicant is owed more than a score.

Overview
🧮 Math & Algorithm
What it is

An adverse-action notice is what fair-lending law requires a lender to give a rejected applicant: the specific, principal reasons their application was declined — not a probability, not "the algorithm decided."

Why it exists — the history
US Equal Credit Opportunity Act (ECOA) / Regulation B
Requires creditors to state specific reasons for denying credit, or tell the applicant they have a right to request them. This predates modern ML entirely — it was written for human underwriters reading paper files — but it applies with equal force to a model-driven decision. A black-box score alone does not satisfy it.

✓ Why this frames the whole module

  • Every technique in Modules 6 — SHAP, LIME, counterfactuals, fairness audits — exists to make this one legal requirement achievable for a model too complex to read directly

✗ What does NOT satisfy this requirement

  • "The model's accuracy is 69%" — a global metric, not a reason for THIS applicant's specific decision
REAL-WORLD FINANCIAL EXAMPLE
RR Finance's baseline classifier rejects a real applicant in Module 6's notebook. Lesson 9 turns that single rejection into an actual adverse-action notice — the direct business deliverable this whole module builds toward.
⬇ Run this lesson in the notebook
No new math in this framing lesson — it sets up why Concepts 2, 3, and 5 (SHAP, LIME, counterfactuals) are the technical answer to a legal requirement, not just academic exercises.
CONCEPT 2

SHAP (Shapley Additive Explanations)

Borrowing a 1953 game-theory idea to split credit fairly among features.

Overview
🧮 Math & Algorithm
What it is

SHAP assigns each feature a numeric contribution to one specific prediction, such that all contributions sum exactly to the gap between that prediction and the model's average output. It is the most theoretically grounded of the explanation methods in this module.

Why it exists — the history
1953 → 2017
Lloyd Shapley devised the Shapley value in cooperative game theory to fairly divide a payout among players based on their marginal contribution across every possible coalition. Lundberg & Lee, 2017, "A Unified Approach to Interpreting Model Predictions" applied this to ML feature attribution, unifying several earlier, less-principled importance methods under one mathematically consistent framework.

✓ Use it when

  • You need explanations with a mathematical consistency guarantee — contributions that provably sum to the actual prediction, not just a plausible-looking ranking
  • You want BOTH a global view (which features matter overall) and a local view (why THIS prediction) from one framework

✗ Watch out for

  • Exact Shapley values are combinatorially expensive (every possible feature subset); real implementations use approximations — KernelExplainer (model-agnostic, slower) or model-specific fast explainers (linear, tree)
REAL-WORLD FINANCIAL EXAMPLE
The notebook explains RR Finance's highest-risk test applicant: SHAP attributes roughly +0.20 of their default-probability estimate to debt_to_income and +0.19 to previous_defaults — real, individually attributable reasons, not a single opaque score.
⬇ Run this lesson in the notebook
φᵢ = Σ_(S⊆F∖{i}) [|S|!·(|F|−|S|−1)!] / |F|! · [f(S∪{i}) − f(S)]
φᵢ is feature i's Shapley value — its fair share of credit for this prediction.
F is the full feature set; S ranges over every subset of the OTHER features.
f(S∪{i}) − f(S) is feature i's marginal contribution when added to coalition S; the formula averages this over every possible coalition, weighted so every ordering of features is equally likely.
Guarantee: Σᵢ φᵢ = f(x) − E[f(X)] — the contributions always sum exactly to "this prediction minus the average prediction." No other common feature-importance method makes this guarantee.
CONCEPT 3

LIME (Local Interpretable Model-agnostic Explanations)

A second, independent way to ask the same local question.

Overview
🧮 Math & Algorithm
What it is

LIME explains one prediction by perturbing the input locally, seeing how the model's output changes, and fitting a simple, interpretable model (weighted linear regression) to approximate the black box just in that local neighbourhood.

Why it exists — the history
2016
Ribeiro, Singh & Guestrin, "Why Should I Trust You?" introduced LIME a year before SHAP, with a simpler, more direct pitch: any black-box model looks approximately linear if you zoom in close enough to one prediction — so fit a linear surrogate right there and read the explanation off it.

✓ Use it when

  • You want a fast, intuitive, model-agnostic local explanation and don't need SHAP's summation guarantee
  • As an independent cross-check against SHAP — genuine agreement between two differently-derived methods is stronger evidence than either alone

✗ Watch out for

  • LIME's local linear surrogate is an approximation with no guarantee its coefficients sum to anything in particular; results can shift between runs unless the perturbation sampling is seeded
REAL-WORLD FINANCIAL EXAMPLE
Run on the exact same RR Finance applicant as the SHAP lesson, LIME independently ranks debt_to_income, previous_defaults, and credit_score at the top too — genuine agreement between two different mathematical approaches, not manufactured.
⬇ Run this lesson in the notebook
ξ(x) = argmin_g L(f, g, πₓ) + Ω(g)
g is the simple surrogate model (typically linear) being fit; f is the real black-box model.
πₓ is a proximity kernel that weights perturbed samples by how close they are to x — nearby perturbations count more, so the surrogate is faithful locally, not globally.
L(f, g, πₓ) is the weighted squared error between the black box's and the surrogate's predictions across the perturbed samples.
Ω(g) penalises surrogate complexity, keeping the explanation itself human-readable (e.g. capping it to a handful of features via num_features).
CONCEPT 4

Explaining a Neural Network

The same SHAP framework, a genuinely different function underneath.

Overview
🧮 Math & Algorithm
What it is

Module 2's ANN is a non-linear, multi-layer function — nothing like the logistic regression's single linear decision boundary. SHAP's model-agnostic KernelExplainer treats it exactly the same way, wrapping it as a black-box predict_proba function and applying the identical Shapley-value machinery from Concept 2.

Why it exists — the history
Same 2017 SHAP paper, model-agnostic branch
KernelExplainer is the general-purpose SHAP estimator that works on any model — the trade-off is speed: it treats the model as a black box and must query it many times per explanation, unlike faster model-specific explainers (linear, tree, or deep-network-specific).

✓ What the notebook demonstrates

  • The exact same applicant, explained by both models — a direct, honest comparison of what each model actually leans on

✗ An honest finding, not smoothed over

  • The reproduced ANN's test AUC trails the logistic regression's; its SHAP explanation for the same applicant partially diverges too — a weaker model isn't "the same reasoning, less accurate," it can genuinely attend to different signal
REAL-WORLD FINANCIAL EXAMPLE
If RR Finance ever ran both models in parallel, this comparison is exactly the check a model-risk-management team would run before trusting either one: do they agree on WHY, not just on WHAT?
⬇ Run this lesson in the notebook
Identical Shapley-value formula to Concept 2 — SHAP's entire design point is that the SAME explanation framework applies regardless of what function sits inside the model.
The only change is computational: KernelExplainer queries the ANN's predict_proba repeatedly on perturbed/masked inputs to estimate each feature's marginal contribution, since it has no access to the ANN's internal gradients the way a deep-learning-specific explainer would.
CONCEPT 5

Counterfactual Explanations

"What would need to change?" — the recourse question SHAP and LIME don't answer.

Overview
🧮 Math & Algorithm
What it is

SHAP and LIME explain why a prediction came out the way it did. A counterfactual explanation answers a different, more actionable question: what is the smallest realistic change to this applicant's inputs that would have flipped the decision?

Why it exists — the history
2017
Wachter, Mittelstadt & Russell, "Counterfactual Explanations Without Opening the Black Box" argued explicitly for counterfactuals as a GDPR-compliant alternative to feature-attribution methods: actionable recourse, not just attribution, is what a person affected by an automated decision actually needs.

✓ Use it when

  • The deliverable is recourse — telling a rejected applicant what would change the outcome — not just a ranked list of important features

✗ An honest limitation, demonstrated directly

  • For a genuinely high-risk applicant, the notebook's search finds NO single-feature change within a realistic range that flips the decision — only a combined two-feature change does. Some applicants simply don't have easy, single-lever recourse, and pretending otherwise would be dishonest
REAL-WORLD FINANCIAL EXAMPLE
The notebook finds that this applicant's decision only flips with a COMBINED change — roughly a 68-point credit-score improvement together with a meaningfully lower debt-to-income ratio — real, specific, actionable guidance a loan officer could actually relay.
⬇ Run this lesson in the notebook
x′ = argmin_(x′) d(x, x′) subject to f(x′) ≠ f(x), x′ realistic
x is the applicant's actual feature vector; x′ is a hypothetical alternative that flips the model's decision.
d(x, x′) measures how large the change is — the notebook uses each feature's own scale (its training-set standard deviation) so a change is judged relative to how much that feature naturally varies.
The notebook's search is a direct, inspectable grid search rather than a continuous optimizer: it tries realistic values (5th–95th percentile of the training data) for each feature, and each pair of features, and reports the smallest change found — an upper bound on the true minimal counterfactual, not a guaranteed global minimum.
CONCEPT 6

Interpretable-by-Design vs. Post-Hoc Explanation

The connecting thread back to Module 5's agent.

Overview
🧮 Math & Algorithm
What it is

Module 5's loan-qualification agent routes between "calculate" and "reject" using an explicit, two-line boolean rule over retrieved policy text. Module 1's classifier decides "approve" or "reject" using millions of internal weight interactions no human can read directly. Both answer "why," but they need fundamentally different explanation strategies.

Why it exists — the history
A standing distinction in interpretable-ML research
Some models are interpretable by design — their decision logic IS the explanation (a rule, a shallow decision tree, a linear model with few features). Others are opaque and need post-hoc tools like SHAP, LIME, or counterfactual search built AFTER training to approximate an explanation the model itself doesn't expose.

✓ What the notebook demonstrates

  • The SAME counterfactual question — "what would need to change?" — asked of both systems: for the agent, the answer is read directly off the rule (two lines of Python); for the classifier, it takes SHAP, LIME, and a grid search across the decision boundary to get the same category of answer

✗ The nuance worth remembering

  • Module 5's router is transparent because THIS implementation chose plain rules for inspectability — a production agent that used an LLM to make the same routing decision would become just as opaque as the classifier, and would need the exact same post-hoc toolkit
REAL-WORLD FINANCIAL EXAMPLE
A real RR Finance deployment will likely mix both kinds of components — some genuinely opaque ML models, some genuinely transparent business rules. Knowing which is which, upfront, determines whether you reach for SHAP or just read the code.
⬇ Run this lesson in the notebook
No shared formula here by design — that is the point of this lesson. The classifier's counterfactual is a numerical optimization/search problem over a continuous, learned decision boundary (Concept 5's formula). The agent's counterfactual is a boolean-logic inversion over an explicit, human-written condition — different mechanics entirely, unified only by the QUESTION being asked, not by any shared mathematics.
CONCEPT 7

Group Fairness & the Age Audit

Finally paying off Module 1's single most-repeated forward reference.

Overview
🧮 Math & Algorithm
What it is

Excluding age from a model's input features prevents DIRECT age discrimination but says nothing about whether the model's outcomes still differ systematically by age group — because other, included features (like employment_years or account_age_months) can act as a correlated proxy. Group fairness metrics measure the actual outcome gap directly.

Why it exists — the history
Fair-lending law, formalised for ML circa 2016–2018
Demographic parity (equal selection rates across groups) and equalized odds (equal true-positive AND false-positive rates across groups) are two of the most widely used formal fairness criteria, popularised for ML audiences by Hardt, Price & Srebro's 2016 "Equality of Opportunity in Supervised Learning" and now implemented in standard toolkits like fairlearn.

✓ What the notebook demonstrates

  • Real, measured demographic-parity and equalized-odds gaps by age group, for BOTH the logistic regression and the ANN — computed with fairlearn.metrics.MetricFrame, not assumed

✗ An honest limitation, stated plainly

  • 800 rows split three ways by age leaves roughly 40–65 test applicants per group — these numbers are directional, not statistically conclusive, exactly the caveat Module 1 attached to its own metrics
REAL-WORLD FINANCIAL EXAMPLE
The audit finds a real, non-zero gap for BOTH models — and the ANN's gap is larger than the logistic regression's on both metrics, on top of its already-lower accuracy. That is exactly the kind of finding a model-governance function exists to catch before a production decision, not after.
⬇ Run this lesson in the notebook
Demographic Parity Difference = max_a P(ŷ=1 | A=a) − min_a P(ŷ=1 | A=a)
Equalized Odds Difference = max( |TPR gap across groups| , |FPR gap across groups| )
A is the sensitive attribute (age group here); ŷ=1 means "predicted default" i.e. rejected.
Demographic parity asks: does the REJECT rate differ by age group, full stop — regardless of whether the rejections are correct.
Equalized odds is stricter: it asks whether the model is EQUALLY ACCURATE across groups — equal true-positive rates (correctly caught defaulters) AND equal false-positive rates (incorrectly rejected non-defaulters) — a model can satisfy demographic parity while still being systematically less accurate for one group.
Both differences are 0.0 at perfect parity; fairlearn and most regulatory guidance treat differences above roughly 0.10–0.20 as worth active investigation, though the exact threshold is a legal/policy judgment, not a fixed universal constant.
CONCEPT 8

Model Governance & Model Cards

Turning scattered findings into one artifact a governance function can actually use.

Overview
🧮 Math & Algorithm
What it is

A model card (or governance report) is a structured, versioned record of what a model is, how it was tested, what its known limitations are, and what its fairness/explainability findings show — built for reviewers, auditors, and future maintainers, not just the data scientist who trained it.

Why it exists — the history
2019
Mitchell et al. (Google), "Model Cards for Model Reporting" proposed a standard template for exactly this: intended use, performance across subgroups, ethical considerations, and known limitations — now a widely adopted pattern across the ML industry for exactly the kind of accountability this module has been building toward.

✓ What the notebook demonstrates

  • A governance report generated PROGRAMMATICALLY from the notebook's own real, computed numbers — never hand-typed, so it can never silently drift out of sync with what was actually found

✗ What this is NOT

  • A marketing document. A real governance report says "here is what we don't know yet" as plainly as "here is what works" — the notebook's report explicitly lists its own sample-size and reproduction-fidelity limitations
REAL-WORLD FINANCIAL EXAMPLE
RR Finance's Module 6 governance report is exactly what a model-risk-management committee would expect to see before approving either the logistic regression or the ANN for any production loan decision.
⬇ Run this lesson in the notebook
No new math — this lesson is a documentation/reporting practice, not a modelling technique. Its value comes entirely from being generated FROM the real numbers computed in Concepts 2–7, not authored separately from them.
CONCEPT 9

Adverse-Action Notices in Practice (Lab)

The direct business deliverable of everything in this module.

Overview
🧮 Math & Algorithm
What it is

The notebook's final lesson assembles SHAP's top reasons (Concept 2) and the counterfactual recourse path (Concept 5) into one templated document: a real adverse-action notice, in the plain language a rejected applicant is legally entitled to receive.

Why it exists — the history
This module's own throughline
Every technique in Module 6 has been building toward exactly this: turning notebook cells into a document a compliance team could actually send. This is the difference between an explainability TECHNIQUE and an explainability WORKFLOW embedded in a real underwriting system.

✓ What the notebook demonstrates

  • A working generate_adverse_action_notice() function that reuses Concept 2 and 5's actual computed values — no new modelling, pure integration of what the module already built

✗ What to try yourself

  • Swap in a different applicant index from the test set and re-run — a genuinely different, applicant-specific notice comes out, because the underlying SHAP and counterfactual computation is real, not templated text
REAL-WORLD FINANCIAL EXAMPLE
This is the actual notice format: predicted outcome, principal reasons ranked by contribution, and either a single-factor or combined-factor path to approval — or an honest "no easy recourse found, refer to manual review" when that's what the data supports.
⬇ Run this lesson in the notebook
No new math — this lesson is pure integration, reusing Concept 2's SHAP values and Concept 5's counterfactual search results directly to populate a human-readable template.

Module 6 hand-off

The notebook explains both the logistic-regression baseline and the reproduced ANN with SHAP and LIME, searches for real counterfactual recourse, finally delivers Module 1's deferred age-fairness audit with fairlearn, connects the classifier's post-hoc explanation needs to Module 5's inherently-interpretable agent routing rule, and closes by generating a real, working adverse-action notice — with a governance report and metrics summary saved as artifacts for the next module.

What Module 7 builds on this
Module 7 (Federated & Privacy-Preserving ML) keeps the same RR Finance classifiers and dataset, but asks what happens when the training data itself must stay private, split across institutions that cannot see each other's records. The explainability and fairness toolkit built here doesn't disappear — a federated or differentially-private model will need the SAME audit before deployment, just computed under a training regime where no single party holds the whole dataset.
←Back to all 12 modules ←Back to Module 5