RR Skillverse · Free Learning Handbook
RR Skillverse Module 10 of 12 Theory Handbook Finance + AI + MLOps

Module 10 — MLOps & On-Premises Deployment

Same theory-first structure as Modules 1–9: for every technique below, what it is, why it exists — the specific problem it was invented to solve — what else you could use instead, and a real financial example. Nine modules have built real, tested models that have only ever run inside a notebook. Module 10 asks how RR Finance actually RUNS one — on its own infrastructure, not a third-party cloud.

⬇ Download the Module 10 notebook ← Back to Module 9 All modules →

Recap — what RR Finance already has

Module 10 is not a fresh start. Here's exactly what it inherits before deploying any of it.

MODULE 1

A real, saved, validated classifier pipeline — the exact model this module wraps in a real running API, unchanged.

MODULE 6

A real fairness and explainability audit — the governance work that has to precede letting this model answer real requests.

MODULE 7

Federated training and a real privacy focus — this module's mTLS and on-premises framing extends that same discipline to the deployment layer.

What's new in Module 10
Every prior module's output was a result in a notebook cell. Module 10 builds the boundary between that and a system a real loan-origination process could actually depend on — a real API, a real container, a real security scan, and a real encrypted network boundary.

From notebook to production

On-premises, specifically — not cloud, and that choice shapes everything in this module.

Why on-premises

RR Finance's applicant data is exactly what Module 7's privacy work and typical financial-sector regulation push toward keeping inside an institution's own infrastructure.

The honest-testing constraint

This sandbox has no Docker daemon, no live Kubernetes cluster, and can't reach Vault's server binary — every real tool is attempted honestly, with disclosed, working alternatives where this sandbox specifically can't reach something.

What ties it together

Every layer in this module — API, container, scan, cluster, secrets, TLS — wraps the SAME model from Module 1, unchanged, all the way through.

CONCEPT 1

FastAPI Model Serving

The boundary between a notebook experiment and a system.

Overview
🧮 Math & Algorithm
What it is

FastAPI, with Pydantic for request validation, wraps a trained model in a real HTTP API — run by uvicorn, a real ASGI server — so it can answer requests from any real caller, not just a notebook cell.

Why it exists — the history
2018
Sebastián Ramírez, FastAPI built on Python's type hints and Starlette/Pydantic to give automatic request validation, serialization, and interactive API documentation essentially for free — quickly becoming one of the most widely used Python API frameworks specifically for ML model serving.

✓ What the notebook demonstrates

  • A real server, started as a real subprocess, wrapping Module 1's ACTUAL saved pipeline — a valid request returns a real prediction (200), an invalid one (credit score 9999) is rejected before it ever reaches the model (422)

✗ What this is NOT

  • A retrained or simplified stand-in model — this is the exact object Module 1 fitted and Module 6 audited, loaded unchanged
REAL-WORLD FINANCIAL EXAMPLE
RR Finance's loan-origination system would call this exact `/predict` endpoint with an applicant's data and get back a real default-probability score and decision — the same computation Module 1's notebook always performed, now reachable over the network.
⬇ Run this lesson in the notebook
No new mathematics — Pydantic's field constraints (ge, le, gt) are simple range checks enforced BEFORE the request reaches the model, not a modelling technique. The model itself is Module 1's already-fitted logistic regression pipeline, unchanged.
CONCEPT 2

Containerization with Docker

Making 'works on my machine' into 'works on their machine.'

Overview
🧮 Math & Algorithm
What it is

A Docker container packages the API, its exact Python dependencies, and the model artifact into one portable, reproducible unit, built from a versioned Dockerfile recipe.

Why it exists — the history
2013
Docker, Inc. popularised Linux containers (building on earlier cgroups/namespaces technology) as a practical, standard way to package and run applications identically across development, testing, and production — replacing "it works on my machine" with a shippable, reproducible artifact.

✓ What the notebook demonstrates

  • A real, valid Dockerfile — with a non-root USER and a HEALTHCHECK already included, deliberately, before Concept 3 shows exactly why those two lines matter

✗ An honest limitation, stated plainly

  • This sandbox has no Docker daemon (no privileged container-in-container access) — the Dockerfile is real and correct, but was never actually built into a running image here
REAL-WORLD FINANCIAL EXAMPLE
RR Finance's infrastructure team would build this exact Dockerfile on their own on-premises servers and run the resulting image identically across every environment — the same file this notebook writes and Concept 3 scans.
⬇ Run this lesson in the notebook
No mathematics here — this is systems engineering. The one structural fact worth naming: a Dockerfile builds in LAYERS, each instruction caching independently, which is why COPY requirements.txt and RUN pip install come before COPY app.py — application code changes far more often than dependencies, so ordering this way avoids re-installing packages on every code change.
CONCEPT 3

DevSecOps Scanning with Trivy

A real automated gate, catching a real, common mistake.

Overview
🧮 Math & Algorithm
What it is

Trivy, downloaded directly from its real GitHub release, scans Dockerfiles (and container images, dependencies, and infrastructure-as-code more broadly) for known security misconfigurations — the automated check a real CI/CD pipeline runs before any container reaches production.

Why it exists — the history
2019
Aqua Security's Trivy became one of the most widely adopted open-source security scanners specifically because it is fast, simple to run, and covers vulnerabilities, misconfigurations, and exposed secrets in one tool — a common default choice in real DevSecOps pipelines.

✓ What the notebook demonstrates

  • A real before/after: a naive Dockerfile fails 2 real checks (HIGH: running as root; LOW: no healthcheck); Concept 2's actual deployment Dockerfile, which already fixed both, scans clean

✗ An honest limitation, stated plainly

  • This sandbox cannot reach mirror.gcr.io, the host Trivy normally pulls its latest vulnerability database and policy bundle from — Trivy handles this gracefully by falling back to its embedded checks, disclosed directly in the notebook's output rather than hidden
REAL-WORLD FINANCIAL EXAMPLE
A container running as root that suffers a container-escape vulnerability hands an attacker root on the HOST machine too — exactly the real, HIGH-severity class of finding Trivy's real scan catches here, on RR Finance's own Dockerfile, before it ever ships.
⬇ Run this lesson in the notebook
No learned model — Trivy's misconfiguration scanning is rule-based policy evaluation (Rego/OPA policies under the hood), similar in spirit to a linter but for infrastructure configuration rather than code syntax. Each rule (like DS-0002, non-root user) is a specific, named, documented check, not a statistical prediction.
CONCEPT 4

Kubernetes Deployment

Surviving a crashed pod, and being reachable by one stable name.

Overview
🧮 Math & Algorithm
What it is

Kubernetes orchestrates multiple copies (replicas) of a containerized service, automatically restarting failed ones and giving them one stable internal address via a Service — a Deployment manifest describes exactly how.

Why it exists — the history
2014
Google open-sourced Kubernetes, building on over a decade of internal container-orchestration experience (Borg), and it became the de facto standard for running containerized services at scale across on-premises and cloud infrastructure alike.

✓ What the notebook demonstrates

  • A real, valid, parseable Kubernetes manifest — 3 replicas, liveness and readiness probes pointed at the SAME /health endpoint Concept 1 actually tested, a non-root security context

✗ An honest limitation, stated plainly

  • This sandbox has no running Kubernetes cluster — the manifest would kubectl apply cleanly on a real one, but was never actually deployed anywhere here, and the notebook says so directly rather than implying otherwise
REAL-WORLD FINANCIAL EXAMPLE
If one of RR Finance's 3 replica pods crashes under load, Kubernetes restarts it automatically using this same manifest — the loan API stays reachable at one stable internal address throughout.
⬇ Run this lesson in the notebook
No mathematics — YAML structure and Kubernetes' own reconciliation loop (continuously comparing the manifest's DESIRED state against the cluster's ACTUAL state, and acting to close any gap) is the underlying mechanism, not a modelling technique. The one worth understanding: livenessProbe and readinessProbe reuse Concept 1's own /health endpoint — a probe failure is what actually TRIGGERS a pod restart or removal from traffic routing.
CONCEPT 5

Secrets Management: From Vault to a Real Local Alternative

Never storing a credential in plaintext.

Overview
🧮 Math & Algorithm
What it is

HashiCorp Vault stores secrets encrypted at rest, behind access policies and an audit log, accessed via a client library rather than a config file — attempted here first via the real hvac client, exactly as this course's discipline requires.

Why it exists — the history
2015
HashiCorp Vault addressed a specific, common real problem: credentials scattered in plaintext across config files, environment variables, and version control — one of the most frequent real causes of financial-sector and enterprise breaches.

✓ What the notebook demonstrates

  • A real, honest attempt to connect to a local Vault server (fails, because Vault's server binary is hosted at releases.hashicorp.com, unreachable here), followed by a real, working AES-based (Fernet) local encryption-at-rest fallback — genuinely unreadable on disk without the key

✗ What the fallback does NOT claim

  • Access-control policies, audit logging, dynamic secret leasing, and multi-node durability are real, substantial Vault capabilities this fallback does not have — only the ONE guarantee actually tested (no plaintext at rest) is claimed
REAL-WORLD FINANCIAL EXAMPLE
RR Finance's database password, instead of sitting in app.py or a Dockerfile in plaintext, is retrieved at runtime from a secrets store — Vault in a real deployment, the notebook's tested local encryption-at-rest fallback here.
⬇ Run this lesson in the notebook
c = E_k(m) where E is AES-128-CBC + HMAC-SHA256 (Fernet)
m is the plaintext secret; k is a symmetric key generated once and stored separately from the encrypted data; c is the ciphertext actually written to disk.
Fernet (used by the notebook's fallback) combines AES-128 in CBC mode for confidentiality with an HMAC for integrity — a real, standard, authenticated encryption scheme, not a custom or weakened one.
The security of this scheme rests entirely on k staying separate from c — exactly the property Vault's real architecture (and this fallback) both depend on.
CONCEPT 6

Mutual TLS

Both sides prove who they are, or the connection never completes.

Overview
🧮 Math & Algorithm
What it is

Mutual TLS (mTLS) requires BOTH the server AND the client to present a valid certificate, signed by a trusted Certificate Authority, before a connection completes — encryption plus two-way authentication, not just one.

Why it exists — the history
TLS: 1999 (RFC 2246) → mTLS as standard internal-service practice, 2010s onward
Standard (one-way) TLS only proves the SERVER's identity to the client — mTLS closes the other half, and became standard practice for internal service-to-service traffic (not just public-facing endpoints) as "zero trust" network architectures became common in the 2010s and 2020s.

✓ What the notebook demonstrates

  • A real X.509 certificate chain (CA, server cert, client cert) generated with real OpenSSL, and a real running HTTPS server that REJECTS a request with no client certificate at the TLS handshake itself, before any application code runs, and ACCEPTS one with a valid client certificate

✗ A real detail this lesson caught directly

  • Modern TLS clients verify hostname against the certificate's SAN (Subject Alternative Name) field, not the legacy CN field alone — a certificate signed without SAN genuinely fails verification in Python's ssl module, exactly the kind of real, specific detail this course surfaces rather than glossing over
REAL-WORLD FINANCIAL EXAMPLE
An internal RR Finance service calling the loan API over mTLS proves its own identity via its client certificate — an attacker who reaches the internal network but has no valid certificate cannot complete a connection at all, regardless of what HTTP request they attempt to send.
⬇ Run this lesson in the notebook
The core mechanism is asymmetric-key signature verification: the CA signs each certificate with its PRIVATE key; anyone holding the CA's PUBLIC key (distributed via ca.crt) can verify that signature, confirming the certificate is genuinely CA-issued without ever needing the CA's private key themselves.
In mTLS specifically, this verification runs TWICE during the handshake: once for the server's certificate (standard TLS) and once for the client's certificate (the "mutual" part) — --ssl-cert-reqs 2 (uvicorn's CERT_REQUIRED) is what makes the client-side check mandatory rather than optional.
Hostname verification (matching the connection target against the certificate's SAN field) is a SEPARATE check from signature verification — a certificate can be validly signed by a trusted CA and still fail verification if its SAN doesn't match the hostname being connected to, exactly the real failure this lesson's build process encountered and fixed.

Module 10 hand-off

The notebook wraps Module 1's real saved pipeline in a real, running FastAPI service, writes a real Trivy-scanned Dockerfile (catching a genuine before/after: 2 real findings, then 0), writes a real, valid Kubernetes manifest, builds a real working secrets-encryption fallback where Vault's server binary is unreachable, and runs a real, tested mutual-TLS boundary around the API -- with a metrics summary saved as an artifact for the next module.

What Module 11 builds on this
Module 11 (AI Cybersecurity) returns to attack the exact system this module just deployed -- STRIDE and MITRE ATLAS threat modelling, model stealing, and adversarial attacks against a model now reachable over a real network boundary, not just inside a notebook. Every deployment choice here (the API's validation layer, the container's non-root user, the mTLS boundary) becomes something Module 11 will specifically try to get around.
←Back to all 12 modules ←Back to Module 9