RAG Step 5: Enhance Your Chat with RAG

Integrate the RAG engine into your Flask chat endpoint so the AI answers questions using your knowledge base.

Step 1: Add the RAG Route

Add a new endpoint to backend/app/routes.py:

from .rag_engine import search_similar_chunks, build_rag_context, generate_rag_response
from .models import Document


@api.route("/rag-chat", methods=["POST"])
def rag_chat():
    data = request.get_json()
    user_message = data.get("message", "")

    if not user_message:
        return jsonify({"error": "Message is required"}), 400

    user_msg = Message(content=user_message, sender="user")
    db.session.add(user_msg)
    db.session.commit()

    try:
        results = search_similar_chunks(db, user_message, Document, top_k=5)
        context = build_rag_context(results)
        ai_reply = generate_rag_response(user_message, context)

        ai_msg = Message(content=ai_reply, sender="ai")
        db.session.add(ai_msg)
        db.session.commit()

        return jsonify({
            "user_message": user_msg.to_dict(),
            "ai_message": ai_msg.to_dict(),
            "evidence": [
                {"title": r["title"], "similarity": round(r["similarity"], 3)}
                for r in results
            ],
        })

    except Exception as e:
        return jsonify({"error": str(e)}), 500

The response now includes an evidence field showing which documents were used.

Step 2: Restart the Backend

Stop and restart with PostgreSQL connected:

docker compose -f advanced/docker-compose.rag.yml up --build

Step 3: Test the RAG Chat

# Ask about tutoring
curl -X POST http://localhost:5000/api/rag-chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Where can I get tutoring at Clark?"}'

Expected response:

{
  "user_message": {"content": "Where can I get tutoring at Clark?", ...},
  "ai_message": {
    "content": "Clark College offers free tutoring in business, English, math, science, and general education classes. You can also access online eTutoring through e-Chat or e-Questions."
  },
  "evidence": [
    {"title": "Tutoring Services at Clark College", "similarity": 0.843},
    {"title": "Tech Hub at Clark College", "similarity": 0.512},
    {"title": "Libraries at Clark College", "similarity": 0.487}
  ]
}

Try more questions:

# Ask about financial aid
curl -X POST http://localhost:5000/api/rag-chat \
  -H "Content-Type: application/json" \
  -d '{"message": "How can I get help paying for college?"}'

# Ask about news
curl -X POST http://localhost:5000/api/rag-chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What happened at the CTE Showcase?"}'

# Ask about events
curl -X POST http://localhost:5000/api/rag-chat \
  -H "Content-Type: application/json" \
  -d '{"message": "Tell me about the Sakura Festival"}'

Step 4: Compare With and Without RAG

RAG off (original /api/chat):

curl -X POST http://localhost:5000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What tutoring is available at Clark?"}'
# → Generic answer: "Most colleges offer tutoring services..."

RAG on (/api/rag-chat):

curl -X POST http://localhost:5000/api/rag-chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What tutoring is available at Clark?"}'
# → Specific answer: "Clark College offers free tutoring in business, English..."

The RAG version gives an accurate, specific answer because it found the tutoring document.

Step 5: How It Works End-to-End

User: "What tutoring is available at Clark?"

1. Embed the question
   "What tutoring is available at Clark?"
   → [0.04, -0.13, 0.29, ...]                 (384-dim vector)

2. Search pgvector
   SELECT * FROM document
   ORDER BY embedding <=> :query_vec
   LIMIT 5

3. Found top 3 similar chunks:
   [0.84] "Tutoring Services at Clark College"
   [0.51] "Tech Hub at Clark College"
   [0.49] "Libraries at Clark College"

4. Inject into prompt:
   System: "Use this context to answer:\n\n
            [Tutoring Services at Clark College]
            Clark College offers free tutoring..."

5. LLM generates answer using context
   "Clark College offers free tutoring in business, English..."

6. Return to user + include evidence

Write Tests for the RAG Pipeline

Test the full flow — embedding, search, and response generation:

from app import create_app, db
from app.models import Document
from advanced.rag.rag_engine import search_similar_chunks, build_rag_context

app = create_app()
with app.app_context():
    results = search_similar_chunks(db, "free tutoring", Document, top_k=3)
    assert len(results) == 3
    assert results[0]["similarity"] > 0.5
    print(f"Top result: {results[0]['title']} (similarity: {results[0]['similarity']:.3f})")

What These Tests Cover

Test Your Work (Manual)

Write a test to verify the RAG pipeline works:

from app import create_app, db
from app.models import Document
from advanced.rag.rag_engine import search_similar_chunks, build_rag_context

app = create_app()
with app.app_context():
    results = search_similar_chunks(db, "free tutoring", Document, top_k=3)
    assert len(results) == 3
    assert results[0]["similarity"] > 0.5
    print(f"Top result: {results[0]['title']} (similarity: {results[0]['similarity']:.3f})")