RR Skillverse Module 11 of 12 Theory Handbook Finance + AI + Security

Module 11 — AI Cybersecurity

Same theory-first structure as Modules 1–10: for every technique below, what it is, why it exists — the specific problem it was invented to solve — what else you could use instead, and a real financial example. The difference now: the moment Module 10 put the model behind a real API, RR Finance acquired a real attack surface. This module threat-models that system and then attacks it — extraction, evasion, and membership inference — measuring every result honestly, including two findings that are deliberately un-dramatic.

⬇ Download the Module 11 notebook ← Back to Module 10 All modules →

Recap — what RR Finance already has, and why it is now a target

Module 11 is not a fresh start. Here is exactly what it inherits — and why that inheritance is now something an adversary can reach.

MODULES 1–6

A validated tabular model, deep-learning fundamentals with adversarial attacks, NLP and a from-scratch LLM, RAG and agents, and an explainability + fairness audit.

MODULES 7–9

Federated and differentially-private training, multimodal pipelines with tamper detection, and graph-based fraud detection.

MODULE 10

A real, running FastAPI service wrapping Module 1's pipeline — Pydantic-validated, containerised, Trivy-scanned, and behind a mutual-TLS boundary. The first time the model was reachable over a network.

What's new in Module 11

Every previous module treated attacks as isolated demos. Module 11 steps back to threat-model the whole deployed system (STRIDE + MITRE ATLAS), then attacks the exact Module 10 API directly — and pairs each attack with a real, tested defence. Two results are kept deliberately undramatic: an attack that fails and a defence that only half-works, because knowing when an attack works is the whole point of thinking like a defender.

Why attack our own system

A model behind an API is only as safe as the weakest thing an attacker can do with query access — and the only way to know that is to try, carefully and honestly.

The core shift

Ten modules built capability; deployment turned that capability into an attack surface. Module 11 measures the exposure that Module 10's convenience created.

The path this module walks

Threat-model the architecture (STRIDE) → map the course to a standard taxonomy (ATLAS) → execute extraction, evasion, and membership inference against the real API → harden it with measured defences.

Why honesty is the method

Every number here comes from a real attack against the real deployed model. Two attacks under-perform — reported as-is, because a security course that only shows scary results teaches nothing about where the real boundaries are.

CONCEPT 1

Threat Modelling with STRIDE

Before attacking anything, enumerate what could go wrong — one structured question per threat class.

Overview

🧮 Math & Algorithm

What it is

STRIDE is a threat-modelling framework that walks six categories of threat — Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege — against a specific system, turning "is this secure?" into six concrete, answerable questions.

Why it exists — the history

1999

Loren Kohnfelder and Praerit Garg at Microsoft introduced STRIDE as a structured way to enumerate threats during design. It remains one of the most widely-taught threat-modelling frameworks precisely because its six categories are exhaustive enough to be useful and few enough to remember.

✓ Use it when

You have a concrete architecture to model — Module 10's API, container, and mTLS boundary give STRIDE something specific to bite on
A security review or regulator expects a documented, category-by-category threat model

✗ Reach for something else when

You need to model attacker capability and likelihood, not just threat categories — STRIDE enumerates what could go wrong, not how likely each is (pair it with a risk-scoring method for that)

REAL-WORLD FINANCIAL EXAMPLE

RR Finance's loan API handles applicant PII and makes money-affecting decisions. STRIDE forces the question 'how could each part of this be spoofed, tampered, or abused?' before an incident, and produces the exact category-by-category table an auditor asks to see.

⬇ Run this lesson in the notebook

STRIDE is a structured enumeration, not a computation — there is no formula. Its rigour comes from completeness: every threat you can name should fall into one of the six categories, so working through all six is a check that nothing was missed.

In the notebook, each category maps to specific Module 10 components and a residual risk after Module 10's real controls: Spoofing → mTLS (strong), Tampering → bounds-only validation (weak), Information disclosure → nothing protecting the model (extraction severe). Lessons 4–6 then execute the three model-specific threats this table identifies.

CONCEPT 2

The MITRE ATLAS Taxonomy

A shared, standard vocabulary for adversarial ML — so the course's scattered attacks become one coherent picture.

Overview

🧮 Math & Algorithm

What it is

MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) is the ML-specific counterpart to the well-known MITRE ATT&CK framework: a real, maintained knowledge base of adversary tactics (goals) and techniques (methods) observed against real ML systems.

Why it exists — the history

2021

MITRE, with industry and academic partners, released ATLAS to give AI security the same shared taxonomy that ATT&CK gave conventional cybersecurity. It catalogues real, documented attacks — data poisoning, model extraction, evasion, and more — as a common language for describing and defending against them.

✓ Use it when

You need to communicate an ML risk to a security team, or compare your exposure against documented industry incidents
You want to turn ad-hoc security demos into an auditable map of which standard techniques you have evidence about

✗ Reach for something else when

You need step-by-step exploit code — ATLAS names and categorises techniques; it is a taxonomy, not an attack toolkit

REAL-WORLD FINANCIAL EXAMPLE

RR Finance's security team cannot triage or defend what it cannot name. Mapping this course's real attacks onto ATLAS turns 'we ran some security demos' into 'here are the seven standard adversary techniques we have real evidence about, and our current control for each' — exactly the governance artifact a review requires.

⬇ Run this lesson in the notebook

No mathematics — this lesson is a mapping over attacks already run. Each entry pairs a real course artifact with an ATLAS tactic and technique.

The notebook maps seven distinct techniques already demonstrated: data poisoning (Module 1), adversarial/evasion crafting (Modules 2, 5), Byzantine model-integrity attack (Module 7), input/document tampering (Module 8), and the three Module 11 attacks — model extraction, query-only evasion, and membership inference.

CONCEPT 3

Model Extraction (Model Stealing)

Reconstructing a functional copy of the model from nothing but its API answers.

Overview

🧮 Math & Algorithm

What it is

Model extraction trains a surrogate model to imitate a target using only (input, output) pairs harvested through the target's API. The attacker chooses inputs, records the returned decisions, and fits their own model — measured by fidelity: how often the surrogate agrees with the victim on fresh inputs.

Why it exists — the history

2016

Tramèr et al., "Stealing Machine Learning Models via Prediction APIs" (USENIX Security) showed that many deployed prediction APIs could be cloned with surprisingly few queries — sometimes recovering model parameters almost exactly. It established model extraction as a practical, not merely theoretical, threat to any model exposed behind an API.

✓ Use it when

You are assessing a model that is exposed behind a query API (like Module 10's /predict) and want to know whether it can be cloned
You want to understand the enabling attack behind offline evasion — a stolen surrogate lets an attacker craft attacks with unlimited, undetected tries

✗ Reach for something else when

The model is never queryable by untrusted parties — extraction needs query access, so a fully internal model with no exposed API is out of scope for this specific attack

REAL-WORLD FINANCIAL EXAMPLE

RR Finance's model is intellectual property and a competitive asset. The notebook shows a linear surrogate reaching very high fidelity from only a few hundred label-only queries — and recovering the victim's actual coefficient structure, i.e. which features drive a rejection and by how much. Module 10's Pydantic validation accepted every query, because each value was in range: validation is not an anti-extraction control.

⬇ Run this lesson in the notebook

The surrogate minimises disagreement with the victim's labels: it fits parameters θ to make surrogate(x; θ) match the victim's returned decision on each queried x. With enough well-spread queries, minimising this disagreement reconstructs the victim's decision boundary.

Query budget — the number of API calls — trades off against fidelity: more queries, closer clone. In the notebook, fidelity climbs with budget and the best (linear) surrogate reaches ~0.99 while recovering the victim's coefficients with Pearson correlation ~0.99 and all 11 signs correct.

Model family matters: the linear surrogate beats tree and forest surrogates here because the victim's boundary is a smooth linear one — a concrete illustration that correctly guessing the victim's architecture makes theft dramatically more effective.

CONCEPT 4

Evasion: Query-Only Adversarial Inputs

Flipping a rejection to an approval by changing only what is reported — with no model internals.

Overview

🧮 Math & Algorithm

What it is

Evasion crafts an input that the model misclassifies in the attacker's favour. In a query-only (black-box) setting the attacker has no gradients — so they start from a real rejected case and greedily probe small, in-range changes, keeping whichever lowers the returned risk, until the decision flips.

Why it exists — the history

2013–2014

Szegedy et al. and Goodfellow et al. showed that tiny, deliberate input perturbations reliably fool neural networks (the adversarial-example result Module 2 demonstrated white-box with FGSM/PGD). Later work extended the attack to the harder black-box setting, where the attacker sees only the model's outputs — exactly the setting a public API creates.

✓ Use it when

You want to test whether a deployed decision can be gamed by an applicant or broker adjusting only reported fields
You need the black-box counterpart to Module 2's white-box gradient attacks — the realistic case when only an API is exposed

✗ Reach for something else when

You control the whole pipeline and can verify every input against an authoritative source — then the attack surface this exploits (self-reported, unverified fields) largely disappears

REAL-WORLD FINANCIAL EXAMPLE

Every successfully evaded rejection is a loan RR Finance's own model judged too risky, approved anyway — direct fraud. The notebook flips a real REJECT to APPROVE in a handful of queries by nudging a single reported field within its legal range; Module 10's bounds-only validation never objects, because each number is individually valid.

⬇ Run this lesson in the notebook

This is a coarse, black-box hill-climb on the model's probability output: at each step, probe each mutable feature by a small fraction of its range, and move in whichever direction most reduces the returned default probability, until the decision crosses the 0.5 threshold.

It is the query-only cousin of Module 2's gradient attacks: gradient methods use ∂loss/∂x directly; here the attacker estimates the useful direction by finite differences — trying a step and observing the output change — because the API exposes no gradient.

What changes the result: a larger step explores faster but overshoots; restricting the mutable set to plausibly-misreportable fields (income, debt, ratios — not lender-verified fields like previous_defaults) keeps the attack realistic. The notebook flips the decision by changing debt_to_income alone.

CONCEPT 5

Membership Inference (and an Honest Failure)

Asking whether a specific person was in the training set — and finding, honestly, that this model does not tell.

Overview

🧮 Math & Algorithm

What it is

Membership inference tries to determine whether a specific record was in a model's training set, using the model's outputs alone. It exploits overfitting: a model that memorised its training data tends to answer more confidently on records it has seen than on ones it hasn't.

Why it exists — the history

2017

Shokri et al., "Membership Inference Attacks Against Machine Learning Models" (IEEE S&P) demonstrated that a model's confidence gap between training and non-training data can leak membership — a genuine privacy breach, since confirming someone was in a sensitive dataset (medical, financial) is itself a disclosure.

✓ Use it when

You are auditing a model's privacy exposure — especially a high-capacity model likely to memorise training rows
You want to test, not assume, whether Module 7's privacy concern actually materialises for a given deployed model

✗ Reach for something else when

The model is a small, well-regularised one that barely overfits — as this lesson shows, there may simply be no membership signal to find, and reporting that honestly is the correct result

REAL-WORLD FINANCIAL EXAMPLE

Confirming a named individual was in RR Finance's default-risk training data leaks that they applied for a loan and were used to model risk — the disclosure Module 7's privacy work aimed to prevent. The notebook runs the attack for real and reports its honest result: it essentially fails on this model.

⬇ Run this lesson in the notebook

The attack scores confidence as the distance of the returned probability from the 0.5 boundary, then measures how well that score separates known members from non-members — reported as an AUC, where 0.5 means the attacker learns nothing.

The attack works only to the extent the confidence distributions differ between members and non-members. That gap exists when a model overfits; it vanishes when a model generalises.

The honest result: the notebook's AUC sits on top of 0.5, because the victim is a simple regularised logistic regression whose train and test AUC are nearly identical (Module 1) — there is no memorisation to leak. The very simplicity that made the model easy to steal (Concept 3) makes it incidentally private. A higher-capacity model would very plausibly leak here where this one does not.

CONCEPT 6

Defences: What Actually Helps, and By How Much

Turning the attack findings into real, measured mitigations — including where a defence only half-works.

Overview

🧮 Math & Algorithm

What it is

Three real controls, each targeting a specific attack: output minimisation (return only the decision, never the probability), rate limiting (cap queries per caller), and input integrity (consistency checks or server-side recomputation of derived features). Each is measured against the live attacks, honestly, including where it falls short.

Why it exists — the history

2015–present

As extraction, evasion, and inference attacks were documented, the defensive literature converged on a defence-in-depth consensus: no single control stops a determined adversary, so real systems layer output minimisation, query throttling, input validation, and downstream anomaly monitoring — accepting that each control raises cost rather than guaranteeing safety.

✓ Use it when

You are hardening a deployed model API and want evidence-backed, prioritised changes rather than a checklist
You need to know the limits of each control — which is exactly what tells you where monitoring must pick up

✗ Reach for something else when

You are looking for a single silver-bullet fix — the honest finding here is that none of these controls, alone, stops the attacks; they raise cost and must be layered

REAL-WORLD FINANCIAL EXAMPLE

A threat model that ends in 'we are exposed' is useless to RR Finance. The notebook produces a prioritised, measured set of changes to Module 10's real service: minimise output (leaking the probability makes theft near-perfect), rate-limit (raises cost but a simple model is cheap to clone), and enforce input consistency (blocks the naive evasion) — while showing that server-side recomputation still lets a consistent lie, like inflated income, through.

⬇ Run this lesson in the notebook

These are engineering controls measured against the real attacks, not new models. Output minimisation is measured as extraction fidelity when the API leaks the probability versus decision-only; rate limiting as fidelity across shrinking query budgets; input integrity by replaying the Lesson 5 evasion against a hardened endpoint.

The subtle result: a tight consistency check does stop the naive evasion (it drove a ratio away from its components), but server-side recomputation does not — the attacker simply reports a higher income, which consistently lowers the recomputed ratios and flips the decision. Input validation can enforce that numbers agree with each other, but not that they are true.

The boundary this exposes: catching an inflated-income lie needs verification against an external source or downstream anomaly monitoring — a control outside the model's request path entirely. The actionable takeaway: minimise output, rate-limit, enforce consistency, and treat the model as stealable and foolable, because over a real API it is both.

Module 11 hand-off

The notebook threat-models Module 10's real architecture, maps the whole course onto MITRE ATLAS, then runs real model-extraction, evasion, and membership-inference attacks against the actual deployed API — reporting each result honestly, including an attack that fails and a defence that only half-works — and saves a threat model, an ATLAS map, and per-attack metrics as artifacts.

What Module 12 builds on this

Module 12 is the capstone: it integrates the whole course into one production system. Module 11's findings become concrete requirements for it — output minimisation and rate limiting on the serving layer, input consistency checks alongside Module 10's range validation, request logging to close the STRIDE Repudiation gap, and a documented, ATLAS-mapped threat model as a governance artifact. The capstone treats the model as an asset an adversary will actively try to steal, fool, and probe.

←Back to all 12 modules ←Back to Module 10