RAG Step 2: Embeddings and Vectors

Before we can search documents, we need to understand how text becomes numbers.

What Is an Embedding?

An embedding is a list of numbers (a "vector") that captures the meaning of a piece of text.

"Cat"     → [0.21, -0.45, 0.83, ...]   (384 numbers)
"Kitten"  → [0.19, -0.42, 0.80, ...]   (very similar to cat!)
"Car"     → [0.55, 0.12, -0.33, ...]   (not similar to cat)

Similar words have similar vectors. The model learns this by reading billions of sentences.

Visualizing Embeddings

Imagine a 3D space where each word is a point:

                    "dog"
                      ●
                     /
                    /
        "cat" ●────● "kitten"
                   /
                  /
    "car" ●───────● "truck"

In reality, embeddings are 384-dimensional (or 768, 1024, etc.) — impossible to visualize, but the math works the same way.

Cosine Similarity

To measure how similar two vectors are, we use cosine similarity:

similarity = cos(θ) = (A · B) / (||A|| × ||B||)

1.0 = identical direction (same meaning)
0.0 = perpendicular (unrelated)
-1.0 = opposite direction (opposite meaning)

"Kitten" vs "Cat":     cosine = 0.92  (very similar)
"Kitten" vs "Car":     cosine = 0.15  (not similar)
"Kitten" vs "Phone":   cosine = 0.08  (not similar)

Word Embeddings vs Sentence Embeddings

Type	Example Model	Input	Output Vector
Word	Word2Vec, GloVe	Single word	1 vector per word
Sentence	`all-MiniLM-L6-v2`	Sentence or paragraph	1 vector for the whole thing

For RAG, we use sentence embeddings — a whole paragraph becomes one vector.

The Model We'll Use: `all-MiniLM-L6-v2`

Size: 80MB (small, runs on any laptop)
Dimensions: 384 (good balance of speed vs accuracy)
Speed: ~10,000 sentences/second on a laptop CPU
Free: No API key needed, runs locally
License: MIT (free for any use)

It's made by the sentence-transformers library from Hugging Face.

Try It

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

sentences = [
    "Clark College offers free tutoring",
    "Students can get help with math",
    "I love pizza",
]

vectors = model.encode(sentences)
print(f"Shape: {vectors.shape}")   # (3, 384)
print(f"Vector 0: {vectors[0][:5]}...")  # First 5 numbers

Output:

Shape: (3, 384)
Vector 0: [0.042, -0.133, 0.287, -0.091, 0.155]...

Why 384 Dimensions?

More dimensions = more nuanced meaning, but slower and more storage.

Dimensions	Model	Storage (1M docs)	Speed
384	all-MiniLM-L6-v2	~1.5 GB	Fast
768	all-mpnet-base-v2	~3 GB	Medium
1536	OpenAI ada-002	~6 GB	Slow (API call)

For this project, 384 dimensions is plenty.

Key Takeaway

Embeddings turn text into coordinates in a "meaning space." Similar meanings have nearby coordinates. Cosine similarity measures the distance. This is how your RAG system will find relevant documents.

← What Is RAG?Setup pgvector →