RAG Step 1: What Is RAG?

RAG stands for Retrieval-Augmented Generation. It's a way to give your AI access to a knowledge base of real documents.

The Problem

LLMs like GPT-4o-mini are trained on public internet data. They don't know about:

Your company's internal documents
Recent news articles (they have a knowledge cutoff)
Private data like student records or support pages

Ask a basic LLM about Clark College tutoring:

User: "Where can I get tutoring at Clark?"
AI: "I'm not sure. Most colleges have tutoring centers."

The AI gives a generic answer because it doesn't have access to Clark's specific information.

The RAG Solution

RAG adds a retrieval step before the LLM call:

User: "Where can I get tutoring at Clark?"

        ┌──────────────────────┐
        │  1. Embed the query  │  ← Convert question to a vector
        └──────────┬───────────┘
                   ▼
        ┌──────────────────────┐
        │  2. Search knowledge  │  ← Find similar documents in vector DB
        │     base for similar  │
        └──────────┬───────────┘
                   ▼
        ┌──────────────────────┐
        │  3. Inject context    │  ← Put retrieved docs into the prompt
        │     into the prompt   │
        └──────────┬───────────┘
                   ▼
        ┌──────────────────────┐
        │  4. Call LLM with     │  ← AI answers using the context
        │     context           │
        └──────────┬───────────┘
                   ▼
        "Clark College offers free tutoring in business, English,
         math, science, and more. You can also get online help
         through eTutoring."

The AI now answers accurately because it has the real information in its context.

How Retrieval Works

Step 1: Embedding — Convert text into a list of numbers (a "vector").

"Clark College offers free tutoring"
            ▼
[0.042, -0.133, 0.287, ...]   (384 numbers)

Step 2: Similarity Search — Find documents whose vectors are closest to the query vector.

Query vector:    [0.1, 0.3, -0.2, ...]
                    │     │     │
Document A:      [0.1, 0.3, -0.2, ...]  ← cosine = 0.98 (very similar!)
Document B:      [0.8, -0.1, 0.4, ...]  ← cosine = 0.23 (not similar)

Step 3: Inject — Take the top 3-5 documents and add them to the prompt.

System: "Use this context to answer: [retrieved documents]"
User:   "Where can I get tutoring at Clark?"

RAG vs Fine-Tuning

	RAG	Fine-Tuning
What it does	Adds documents to prompt	Retrains the model
Cost	Free (just prompt tokens)	Expensive ($10-$100+)
Update speed	Instant (add a document)	Days (retrain)
Accuracy	High (exact docs)	Medium (model memorizes)
Best for	Facts, policies, news	Tone, style, behavior

For this project, we use RAG — it's simpler, cheaper, and lets you update the knowledge base by just adding documents.

What You'll Build

In the next steps:

Embeddings and vectors — understand how text becomes numbers
Set up pgvector — PostgreSQL with vector search
Ingest knowledge — load Clark College news and support articles
RAG chat enhancement — add retrieval to your chat app

← Solutions Embeddings →