Module 12 — Final Capstone: CAMEL Sentinel
Eleven modules taught each technique on the synthetic RR Finance teaching system, so the mechanics were always clear. The capstone does the harder thing: it applies that same twelve-stage path to real public FDIC bank data, and wires everything — model, explanations, fairness audit, guard-railed assistant, fraud-detecting GNN, an AI security lab, and one-command deployment — into a single, defended, runnable product. This page is a guided tour of that system and how each earlier module shows up inside it.
Recap — everything the course built
The capstone assumes all eleven modules. Here is the ground it stands on.
A validated tabular model, deep-learning fundamentals with adversarial attacks, financial NLP with PII redaction, and an aligned from-scratch language model with red-teaming.
Retrieval and tool-using agents, explainability and a fairness audit, federated and differentially-private training, and multimodal document and voice intelligence.
Graph-based fraud detection, a real containerised and mutual-TLS-protected API deployment, and an AI-security module that attacked that deployment and hardened it.
From a teaching set to a real product
An honest note on what changes when the training wheels come off.
The data gets real
Modules 1–11 used the synthetic RR Finance loan set so every mechanism was inspectable. The capstone trains on 265,114 real FDIC bank-quarters (2005–2023) — imbalanced, messy, and validated out-of-time on the untouched 2012–2023 years.
The pieces become one system
Model, explanations, fairness audit, assistant, fraud GNN, security lab and deployment stop being separate notebooks and become one FastAPI service behind one UI, in one Docker image.
The honesty stays
The capstone keeps the course's discipline: PR-AUC over accuracy for rare failures, an out-of-time test set, disclosed synthetic data where used, and a security posture that names its residual findings instead of hiding them.
Where each module lives in the capstone
Every one of the eleven modules is used where it genuinely fits — not bolted on for completeness.
| # | Module | Where it lives in CAMEL Sentinel |
|---|---|---|
| 1 | Data Science & Financial Data Analysis | FDIC pipeline, CAMEL ratios, EDA, baseline + tuned models |
| 2 | Deep Neural Networks | Layers, backprop and Adam inside the GNN; adversarial-example ideas in the Security Lab |
| 3 | NLP & Financial Text AI | Prompt-injection detection and PII redaction in the assistant |
| 4 | Generative AI, LLMs & RLHF | Chat assistant with server-side guardrails |
| 5 | RAG, LangChain & AI Agents | Allow-listed score_bank tool and tool-calling agent |
| 6 | Explainable & Responsible AI | SHAP, LIME, counterfactuals, fairness audit, model card |
| 7 | Federated & Privacy-Preserving ML | FedAvg simulation with clipping and noise |
| 8 | Multimodal AI | Voice input/output; deepfake-voice lab |
| 9 | Graph Neural Networks | GraphSAGE fraud-ring detection |
| 10 | MLOps & Deployment | FastAPI, Docker, Docker Hub, ACR, Azure Container Apps, Kubernetes, Trivy |
| 11 | AI Cybersecurity | AI Security Lab, STRIDE, MITRE ATLAS, Security Posture |
One System, Cleanly Separated
A browser talks to a UI, the UI talks HTTP to an API, and only the API touches the models — three tiers that can be tested, scaled and replaced independently.
The capstone, CAMEL Sentinel, is a single deployable product with a strict separation of concerns: a Streamlit UI that never loads a model, a FastAPI service that owns all the intelligence (the risk model, the fraud GNN, the security-lab engine, the guard-railed assistant), and an optional Azure OpenAI/Speech provider the API calls only when chat is switched on. One Docker image contains everything; a MODE switch decides whether it runs as one container or as separate UI and API services.
✓ Why it's built this way
- The UI can be pointed at any CAMEL API (local or cloud) without code changes — a real backend-switch control, allow-listed and SSRF-safe
- Each tier is independently testable; the model can be upgraded without touching the UI
✗ What it deliberately avoids
- The UI holding model logic — that would couple presentation to inference and make the API pointless
- The LLM making decisions — risk numbers come only from the ML model, never invented by the assistant
MODE=all|api|ui) is the container discipline Module 10 introduced with Docker and Kubernetes.An Early-Warning Model on Real Data
The same modelling discipline the course taught on a synthetic set, now applied to 265,114 real FDIC bank-quarters.
Banks are rated on CAMEL(S) — Capital, Asset quality, Management, Earnings, Liquidity (and Sensitivity). Real ratings are confidential, so the capstone learns an early-warning signal from public data: six financial ratios from 265,114 real FDIC bank-quarters (2005–2023), with 4,165 "failed within two years" examples, predicting failure risk. The model is a tuned HistGradientBoostingClassifier, chosen from ten compared families (RandomForest, XGBoost, LightGBM, ExtraTrees, logistic regression, imbalanced-learn ensembles, and more).
✓ How it is validated honestly
- Tuned only on 2005–2011, then tested on the untouched 2012–2023 period — a real out-of-time split, not a random shuffle
- PR-AUC is the headline metric (54.9%) because failures are rare; ROC-AUC (97.3%) is reported alongside, not instead
✗ What it is not
- Not an official CAMELS supervisory rating, and not financial advice — a relative early-warning ranking from public statement proxies
- Not evaluated on the same years it was tuned on — the trap the course warned about repeatedly
Explaining and Auditing Every Decision
A score no one can interrogate is a liability; the capstone explains each decision three ways and audits the model for fairness.
For every bank it scores, the capstone shows why: a SHAP breakdown (global and per-bank), a LIME cross-check from a second method, and a bounded counterfactual search that answers "what would have to change for this bank to be judged safe?". On top of that sits a Responsible-AI audit — subgroup performance by asset size and region — and a published model card documenting intended use, data, and caveats.
✓ What this buys
- A decision a regulator or a bank officer can actually inspect, not a black-box number
- Two independent explanation methods that can disagree — surfacing when an explanation is fragile
✗ What it refuses to do
- Present a score without its reasons
- Ship without a fairness audit or a model card — governance is part of the artifact, not an afterthought
An Assistant That Cannot Lie About Risk
A language model people can chat with, wrapped so it can only ever explain the real model's numbers — never invent them.
The capstone includes an optional LLM assistant (text and voice) that lets a user ask about a bank in plain English. Crucially, the language model is a translator, not a decision-maker: risk numbers reach it only through a single allow-listed score_bank tool with validated arguments. Around it sit real guardrails — a prompt-injection pre-check, PII redaction before anything leaves for the provider, and sanitised output — so the assistant can explain the model's reasoning without ever being able to fabricate a rating.
✓ The guardrails that make it safe
- An allow-listed tool with validated arguments is the only path to a number — the LLM can't route around it
- PII is redacted before the provider sees it; injection is checked before the prompt is trusted
✗ What it is not allowed to be
- A source of risk scores in its own right — it may only relay what the ML model returned
- A channel that leaks user PII to a third-party provider, or that trusts injected instructions
score_bank tool and tool-calling agent are Module 5; voice input/output (and the deepfake-voice lab) are Module 8.Catching Fraud Rings a Tabular Model Misses
Some risk lives in the relationships between accounts, not in any single row — so the capstone adds a graph model.
Alongside the tabular risk model, the capstone runs a heterogeneous GraphSAGE graph neural network on a (clearly-labelled synthetic) transaction graph, and compares it against tabular baselines on the same data. The point is pedagogical and real: a fraud ring — a pattern of connections between accounts — is invisible to a model that sees each account in isolation, but visible to one that learns from the graph structure.
✓ What the graph model adds
- Detection of ring-shaped fraud patterns that per-row tabular models structurally cannot see
- A like-for-like comparison against tabular baselines, so the graph model's advantage is measured, not asserted
✗ Where it is honest about limits
- The transaction graph is synthetic and labelled as such — the technique is real, the data is a teaching stand-in
- It complements, not replaces, the tabular model
Attacking and Defending the App Itself
The security modules end where they should: a lab that attacks this very application, and shows the defence for each attack.
The capstone ships an AI Security Lab: a STRIDE + MITRE ATLAS threat map of the app, and ten attack → defence labs run against the app itself — prompt injection, model stealing, adversarial examples on the real model, deepfakes, PII leakage, data poisoning, and more. A separate Security Posture page documents how the app protects itself and includes a live self-test, backed by real scans: Trivy config 27/27 pass and 0 Python-package CVEs in the image.
✓ Why this is the honest ending
- Every attack is run against the real running app, and paired with the actual control that stops it
- Security is measured (Trivy results, a live self-test), not claimed
✗ What it doesn't pretend
- That the app is unbreakable — remaining OS-level findings with no upstream fix are disclosed, not hidden
- That a single control is enough — defence in depth, exactly as Module 11 concluded
Shipping It Anywhere as One Container
The whole system — UI, API, and models — packaged as a single hardened image you can run in one command.
The capstone is delivered as one Docker image on Docker Hub (raushanranjan/camel-sentinel), runnable with a single docker run, and deployable to Azure Container Apps, a Kubernetes cluster (manifests included), or Docker Compose as two services. The image runs as a non-root user, bakes in no secrets, pins its dependencies, and ships without curl or pip at run time — the production-shaped container Module 10 described, hardened by Module 11's findings.
✓ What "production-shaped" means here
- One command to run the entire system; models and data are included, so there is nothing to train first
- Non-root, no baked-in secrets, pinned dependencies, and a health check — verified by Trivy
✗ What it refuses to ship
- Secrets in the image or echoed in errors — keys are supplied at run time via environment only
- An image that assumes trust — CORS is allow-listed and the API is private by default
Run the whole thing yourself
The models and data are included in the image, so there is nothing to train first — one command brings up the entire system.
docker run -d --name camel -p 8501:8501 raushanranjan/camel-sentinel:latesthttp://localhost:8501. Wait ~1–2 minutes for the health check to go green while the models load, then explore: Score a Bank (with SHAP, LIME and counterfactuals), the Fraud GNN, the AI Security Lab, and the Security Posture self-test. Scoring, explanations, fraud detection, the security lab and the roadmap all work with no keys; only the chat assistant needs optional Azure keys.You've finished the course
Twelve modules ago this was a single tabular model on a synthetic dataset. You now understand — and can point to a running, defended system that demonstrates — the full path from raw financial data to a deployed, explainable, secured AI product.