RAG Challenges

Try these after completing all 5 RAG steps.

Knowledge Check

What is the difference between RAG and fine-tuning?
Why do we chunk documents before embedding them?
What does cosine similarity measure?
Why are there 384 dimensions in the all-MiniLM-L6-v2 model?
What happens if the knowledge base doesn't contain a relevant document?

Code Challenges

Challenge 1: Switch Embedding Model

Replace all-MiniLM-L6-v2 with a different model like all-mpnet-base-v2 (768 dimensions). Update the Document model to use Vector(768) and re-index. Compare the search quality.

Challenge 2: Hybrid Search

Combine vector search with keyword search. For example, use PostgreSQL full-text search (tsvector) as a fallback. Show results from both, then merge and deduplicate.

Hint:

SELECT * FROM document
WHERE to_tsvector('english', content) @@ plainto_tsquery('english', :query)

Challenge 3: Reranking

After retrieving the top 20 results from pgvector, use a cross-encoder model (like cross-encoder/ms-marco-MiniLM-L-6-v2) to rerank them. Return the top 5 after reranking.

Challenge 4: Web Scraper

Build a scraper that fetches the latest Clark College news and adds them to the knowledge base automatically:

import requests
from bs4 import BeautifulSoup

url = "https://www.clark.edu/news-and-media/24-7/"
# Parse page, extract articles, add to DOCUMENTS

Challenge 5: RAG History

Modify the /api/rag-chat endpoint to include the last 5 messages in context (not just the latest question). This lets the AI ask follow-up questions.

Open-Ended Project: Semantic Cache

Build a cache that stores embeddings of common questions and their answers. Before calling the LLM, check if a similar question has been asked before:

def check_cache(db, query, threshold=0.95):
    """Return cached answer if a very similar question exists."""
    query_vec = embed_text(query)
    cached = (
        db.session.query(CachedQA)
        .order_by(CachedQA.question_embedding.cosine_distance(query_vec))
        .limit(1)
        .first()
    )
    if cached and (1 - cached.question_embedding.cosine_distance(query_vec)) > threshold:
        return cached.answer
    return None

This saves API calls and speeds up responses for common questions.

← Chat Enhancement Solutions →