RR Skillverse · Free Learning Handbook
RR Skillverse Module 5 of 12 Theory Handbook Finance + AI + Security

Module 5 — RAG, LangChain & AI Agents

Same theory-first structure as Modules 1–4: for every technique below, what it is, why it exists — the specific problem it was invented to solve — what else you could use instead, and a real financial example. Module 4 built an aligned language model with a hard limit: it only knows what was baked into its weights. Module 5 gives it the ability to look things up and take real actions.

⬇ Download the Module 5 notebook ← Back to Module 4 All modules →

Recap — what RR Finance already has

Module 5 is not a fresh start. Here's exactly what it inherits before adding retrieval and agentic tool-use.

MODULE 1–2

A validated tabular baseline and deep-learning fundamentals, including the first inference-time attack (adversarial examples).

MODULE 3

Text representations and contextual embeddings — the same similarity math this module's retrieval lessons build directly on.

MODULE 4

A real, aligned, instruction-following model built from scratch (SFT → Reward Model → RL/DPO → LoRA), plus real LLM red-teaming.

What's new in Module 5
Module 4's model is powerful but "closed-book" — frozen knowledge, no access to current data, no ability to act. Module 5 opens the book (retrieval) and gives the model hands (tools) — then closes with the new attack surface that comes with both.

Why retrieval, why agents

A model that only recalls what it memorised during training will confidently make things up when asked about anything that changed since — the problem this whole module solves in two connected halves.

The core problem

Parametric knowledge is frozen at training time — a policy that changed yesterday isn't in the weights, no matter how large the model is.

The pipeline this module walks

Chunk documents → embed them → retrieve by similarity → augment a prompt with what's retrieved → generalise into a tool-using, multi-step agent.

Why the security lessons follow naturally

A system that trusts retrieved content, and can act on it, is a system whose retrieved content can manipulate its actions — the natural next step after Module 3/4's prompt injection and jailbreaks.

CONCEPT 1

Why RAG: Parametric Knowledge Has a Hard Limit

A frozen model plus a live document store beats retraining every time a policy changes.

Overview
🧮 Math & Algorithm
What it is

Retrieval-Augmented Generation (RAG) keeps a model's weights frozen and instead retrieves relevant real documents at query time, handing them to the model as context — "open-book" answering instead of "closed-book" recall from memorised weights.

Why it exists — the history
2020
Lewis et al., Facebook AI, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" combined a pretrained retriever with a pretrained generator, trained jointly. Most production RAG systems today use a simpler, decoupled version of the same idea — a separate retriever feeding context into an off-the-shelf LLM — which is what this module builds.

✓ Use it when

  • Knowledge changes faster than you can retrain (RR Finance's policies, rate sheets)
  • Answers must cite specific, current, or private documents a model shouldn't have memorised into its weights

✗ Reach for something else when

  • The knowledge is genuinely stable and general (Module 4's fine-tuning is still the right tool for teaching a model a new skill or style, not new facts)
REAL-WORLD FINANCIAL EXAMPLE
RR Finance's loan policies change constantly. Retraining Module 4's model every time a rate sheet updates would be slow and expensive — RAG instead retrieves the current policy document at answer time, so the model is always working from today's real numbers.
⬇ Run this lesson in the notebook
Lesson 1 is conceptual — the retrieval math (embeddings, cosine similarity, BM25 scoring) is built in Lessons 3–5.
top_k is the one parameter introduced here: how many retrieved chunks get included in the prompt. Too few and relevant context is missed; too many dilutes the prompt with irrelevant text and wastes context budget — the same precision/recall trade-off that shows up throughout this handbook.
CONCEPT 2

Chunking a Document Corpus

Splitting long documents into retrievable, self-contained pieces.

Overview
🧮 Math & Algorithm
What it is

Whole documents are usually too long to retrieve and hand to a model wholesale. Chunking splits documents into smaller, retrievable pieces — small enough to be precise, large enough to retain context.

Why it exists — the history
Standard practice
Fixed-size chunking with overlap is the simplest, most widely used approach in production RAG systems: split on a target size, keep a small overlap between consecutive chunks so a fact sitting near a boundary doesn't get cut in half and lost.

✓ Use it when

  • Documents are longer than a retrievable unit should be (most real policy documents, manuals, filings)

✗ Watch out for

  • Too small a chunk_size fragments a single fact across multiple chunks; too large re-introduces the "too much irrelevant text" problem chunking was meant to solve
REAL-WORLD FINANCIAL EXAMPLE
RR Finance's Personal Loan policy document gets split into self-contained chunks — one about maximum amount and rates, another about credit score requirements — so a query about credit scores retrieves precisely that chunk, not the entire policy document.
⬇ Run this lesson in the notebook
chunk_size, chunk_overlap (characters or tokens)
Purely an information-design choice, not a learned model — RecursiveCharacterTextSplitter tries to split on natural boundaries (paragraph, then sentence, then word) before falling back to a hard character cut, to avoid severing a sentence mid-word wherever possible.
chunk_overlap is a safety margin: consecutive chunks share a few dozen characters, so a fact sitting exactly on a chunk boundary still appears whole in at least one of the two adjacent chunks.
CONCEPT 3

Embeddings for Retrieval

Turning a chunk of text into a vector that captures its meaning.

Overview
🧮 Math & Algorithm
What it is

A sentence embedding model maps a whole chunk of text to one dense vector, trained so semantically similar text ends up with similar vectors — a direct evolution of Module 3's Word2Vec (per-word) and contextual embeddings (per-token) to the chunk/document level.

Why it exists — the history
2019
Reimers & Gurevych, "Sentence-BERT" fine-tuned BERT-style models specifically to produce comparable sentence-level vectors — dramatically faster than comparing every pair of sentences with a full attention pass every time.

✓ Use it when

  • You need to catch paraphrases — a query about "loan limit" should find a chunk saying "maximum amount," with no words in common

✗ Reach for something else when

  • Real internet access isn't available — this notebook falls back to Module 3's TF-IDF vectors, which are sparse and keyword-based rather than dense and semantic, and honestly weaker at catching paraphrases
REAL-WORLD FINANCIAL EXAMPLE
Customers and staff phrase questions differently than policy documents are written. Embedding-based retrieval catches "how much can I borrow" matching a chunk that says "maximum amount" — pure keyword search would miss it entirely.
⬇ Run this lesson in the notebook
cosine_similarity(q, c) = (q·c) / (‖q‖‖c‖)
Same cosine similarity formula as Module 3's Word2Vec lesson, applied to whole-chunk vectors instead of single-word vectors: 1 means same direction (highly relevant), 0 means unrelated.
Embedding dimensionality (e.g. 384 for all-MiniLM-L6-v2) is fixed by the chosen model — higher dimensionality can capture more nuance at the cost of more storage and compute per comparison.
The TF-IDF fallback is technically also an "embedding" in the loose sense (a vector representation of text) — just sparse (mostly zeros, one dimension per vocabulary word) and keyword-based rather than dense and semantic.
CONCEPT 5

Lexical Retrieval With BM25, and Hybrid Search

Why exact keyword match still matters alongside semantic search.

Overview
🧮 Math & Algorithm
What it is

BM25 (Best Matching 25) is a classic, still widely used lexical ranking function — an evolution of Module 3's TF-IDF that additionally accounts for document length and saturates term-frequency's contribution.

Why it exists — the history
1994
Developed by Robertson, Walker et al. as part of the Okapi information-retrieval system — still the standard lexical baseline decades later, and often run alongside vector search in production ("hybrid" retrieval), since neither approach dominates the other across all query types.

✓ Use it when

  • Exact, rare terms matter — a specific policy code, an exact numeric threshold, a compliance clause number

✗ Reach for something else when

  • The query is a paraphrase with no shared vocabulary with the target chunk — that's exactly where dense embeddings (Concept 3) win instead
REAL-WORLD FINANCIAL EXAMPLE
A compliance officer searching for an exact clause number or a specific rate value cares about exact-match precision more than semantic similarity — this notebook demonstrates BM25 and vector search returning genuinely different top results for the same query.
⬇ Run this lesson in the notebook
score(q,D) = Σ IDF(t) × [tf(t,D)×(k1+1)] / [tf(t,D) + k1×(1−b+b×|D|/avgdl)]
IDF(t) — the same distinctiveness idea as TF-IDF (Module 3): common terms contribute little, rare terms contribute more.
k1 controls how quickly term-frequency saturates — a term appearing 10 times isn't 10× as relevant as it appearing once; the denominator's tf + k1 term flattens that curve.
b controls how much document length is penalised — a term match in a short chunk counts for more than the same match in a long one, via the |D|/avgdl length-normalisation term.
CONCEPT 6

A Full RAG Pipeline With LangChain

Composing retrieval, prompting, and generation with the industry-standard framework.

Overview
🧮 Math & Algorithm
What it is

LangChain's Runnable interface lets you compose a retriever and an LLM into a single callable pipeline (retriever | prompt | llm) — the same retrieval mechanism as Concept 4, now wrapped in a real, industry-standard orchestration layer instead of hand-wired glue code.

Why it exists — the history
2022
LangChain (Harrison Chase) emerged as the LLM ecosystem exploded, standardising the "retriever + prompt + LLM" pattern so teams stop rebuilding the same plumbing (retries, prompt templating, swappable components) repeatedly.

✓ Why the framework matters

  • Swap the local template-based responder this notebook uses for Module 4's fine-tuned model, or a hosted LLM API, and the SAME pipeline code works unchanged

✗ What's simplified here, on purpose

  • The generation step uses a lightweight local template responder, not a full LLM — clearly marked as the one piece you'd swap for production
REAL-WORLD FINANCIAL EXAMPLE
A production RAG system needs retries, prompt templating, and observability so RR Finance's engineering team doesn't rebuild all of it from scratch — LangChain provides that layer on top of the retrieval math already built in Concepts 3–5.
⬇ Run this lesson in the notebook
No new math in this lesson — it's about real software architecture for the retrieval math already built. The PromptTemplate controls exactly how retrieved chunks are formatted into the model's context, and real RAG answer quality is highly sensitive to that template's exact wording.
CONCEPT 7

AI Agents: Giving the Model Tools

From reading documents to taking real actions.

Overview
🧮 Math & Algorithm
What it is

An agent is a model that can call functions ("tools") and use their results. RAG lets a model read; tool-use lets it check live data and perform real actions — calculate an EMI, look up a balance, flag a case.

Why it exists — the history
June 2023
Function-calling/tool-use for LLMs became mainstream with OpenAI's function-calling API, quickly adopted industry-wide (including by Anthropic and Google) as the standard way to connect a language model to real actions and live data.

✓ Use it when

  • The task needs current, live data (not just retrievable documents) or a real action (a calculation, a lookup, a state change)

✗ Watch out for

  • Giving a model access to a dangerous tool (fund transfers) with the same casual trust as a safe one (a calculator) creates exactly the risk Concept 9's poisoning demo exploits
REAL-WORLD FINANCIAL EXAMPLE
A useful RR Finance assistant needs to calculate a real EMI or look up a current policy, not just recite retrieved text — this notebook's agent chooses between a calculate_emi tool and a lookup_policy tool based on the request.
⬇ Run this lesson in the notebook
Tool selection is a classification decision — which tool, if any, fits this request — made by the same kind of model Module 4 built, just prompted (or, in this notebook's simplified router, rule-matched) to output a structured tool call instead of free text.
Each tool's description is what a real LLM-based router uses to decide when to call it — vague descriptions lead to picking the wrong tool, or missing an opportunity to use one at all.
CONCEPT 8

LangGraph: Multi-Step, Stateful Agent Workflows

Explicit, inspectable graphs instead of implicit agent loops.

Overview
🧮 Math & Algorithm
What it is

LangGraph models an agent as an explicit state graph: nodes are steps (retrieve, decide, calculate, respond), edges — including conditional ones — define what happens next based on the current state.

Why it exists — the history
2024
Built by the LangChain team specifically because chaining tool calls implicitly (Concept 7's approach, and LangChain's earlier AgentExecutor) becomes hard to control and debug once workflows have conditional branches or loops — an explicit graph is easier to reason about, test, and visualise.

✓ Use it when

  • The task needs multiple conditional steps in sequence — "check eligibility, and if it passes calculate the EMI, and if not explain why" is a workflow a single tool call can't express

✗ Overkill when

  • A single tool call answers the request — Concept 7's simpler router is enough, and adding graph machinery just adds complexity
REAL-WORLD FINANCIAL EXAMPLE
This notebook's graph checks loan qualification against a retrieved policy, then routes to either a calculation step or a rejection step — a real, inspectable, multi-step workflow instead of one flat function.
⬇ Run this lesson in the notebook
Graph = (Nodes, Edges); conditional_edge(state) → next_node
The state object is passed between every node and can be read/updated by each one — the shared memory of the whole workflow, closer to classical graph traversal than to any learned model.
A conditional edge that routes back to an earlier node produces looping/retry behaviour — the graph structure directly controls what workflows are even expressible, which a flat sequential function cannot represent.
CONCEPT 9

Cybersecurity: Retrieval Poisoning

When an attacker gets a malicious document into the document store.

Overview
🧮 Math & Algorithm
What it is

A RAG system trusts whatever is in its document store. If an attacker gets a malicious document INTO that store, the retriever will happily surface it as if it were legitimate policy — the retrieval-era evolution of Module 3's prompt injection.

Why it exists — the history
Direct consequence of RAG's design
Any system that indexes and retrieves from a document collection inherits this risk the moment that collection accepts content from a less-trusted source — a shared drive, a support ticket, a customer upload — the same "don't trust user-controllable content" lesson Module 3 and 4 taught, now applied to the retrieval layer.

✓ What the notebook demonstrates

  • A real, working attack — a crafted document engineered to rank highly against a predictable query, then shown to actually get retrieved ahead of the genuine policy chunk

✗ Real defences

  • Access control on who can add documents, provenance/trust scoring per source, and treating retrieved content as untrusted data to verify — never as an implicit instruction
REAL-WORLD FINANCIAL EXAMPLE
If RR Finance's document ingestion pipeline accepts uploads from a less-trusted source, a poisoned "policy update" sitting there could get retrieved into a real customer-facing answer — exactly what this notebook's attack demonstrates working.
⬇ Run this lesson in the notebook
The attack simply exploits the retrieval math already built (Concepts 3–4): an attacker who knows or guesses likely customer queries can hand-craft text that maximises cosine similarity to those queries — the retrieval-layer version of Module 3's TF-IDF keyword-stuffing evasion attack.
top_k (Concept 1) directly controls blast radius: a smaller top_k means a poisoned document must rank very highly to get included at all; a larger top_k gives an attacker more room to sneak in.
CONCEPT 10

Cybersecurity: Tool-Call Hijacking

When retrieved content manipulates what an agent does, not just what it says.

Overview
🧮 Math & Algorithm
What it is

Concept 9 poisoned what the agent reads. This is the higher-stakes version: a malicious retrieved document that manipulates an agent into calling a tool it shouldn't, or with attacker-chosen arguments — an agent with real tool access that trusts retrieved content the same way it trusts direct user input.

Why it exists — the history
2023–2024
As agentic LLM systems with real tool access became common, security researchers identified indirect prompt injection via retrieved content as a distinct, more dangerous variant of Module 3's direct prompt injection — "indirect" because the attacker never talks to the model directly, only plants content the model will later retrieve and trust.

✓ What the notebook demonstrates

  • A working hijack — a routine-looking case note with an embedded instruction gets retrieved for an unrelated query, and its text triggers an unintended, fraudulent tool call

✗ The real fix

  • A trust boundary: treat retrieved document content as DATA to reason about, never as INSTRUCTIONS to act on — tool-routing decisions should come ONLY from the actual user's request, demonstrated working in the notebook
REAL-WORLD FINANCIAL EXAMPLE
An agent with access to real tools (fund transfers, account changes, approvals) that can be manipulated via retrieved content is a direct fraud vector — the highest-stakes risk in this entire module, and the reason production agent systems need an explicit trust boundary between retrieved data and actionable instructions.
⬇ Run this lesson in the notebook
No new math — like Concept 9, this is a systems/trust-boundary issue, not a modelling one. Any agent that lets retrieved content influence tool-selection logic the same way user input does has this vulnerability, regardless of whether that logic is keyword rules (this notebook, for full inspectability) or an LLM-based router (a real deployment) — the underlying flaw is identical either way.

Module 5 hand-off

The notebook chunks and embeds a real RR Finance policy corpus, builds real FAISS and BM25 indexes, composes a real LangChain RAG pipeline, wires a tool-using agent through a real LangGraph workflow, and produces two working security demonstrations — saved as artifacts and a metrics summary for the next module.

What Module 6 builds on this
Module 6 (Explainable & Responsible AI) turns the lens back on every model built so far — including this module's retrieval and agent decisions — asking not just "does it work?" but "can we explain why it made this specific decision, and is that decision fair?" Module 1's deferred age-fairness question finally gets its full audit here, using SHAP, LIME, and counterfactual explanations.
←Back to all 12 modules ←Back to Module 4