Why I Went Hybrid: My First RAG Project

While prepping to build my first AI chatbot, I kept reading about RAG — Retrieval-Augmented Generation. Sounded great in theory. Then I dug into how vector search actually works and realized it wasn't going to be enough on its own.

What I built

A small chat app that talks to an Azure AI Foundry Agent through a Go backend, with a React frontend on top. Nothing too fancy on the surface — but under the hood, it uses Hybrid RAG to actually find the right stuff in my documents. That means PostgreSQL with pgvector for vector similarity, combined with full-text keyword search. And to keep things cheap, embeddings run locally using all-MiniLM-L6-v2 — no external API calls needed for that part.

The problem with pure vector search

Here's the thing about vector search: it's amazing at understanding meaning. Ask it something conceptual and it'll find related content even if the exact words don't match. Love that.

But try searching for something specific — a config parameter name, an error code, a particular tool or library — and suddenly it gets... fuzzy. It returns stuff that's kind of related but misses the document that literally contains the exact term you typed. That's frustrating when you know the answer is in there.

I ran into this over and over. I'd ask about pgvector and get back general paragraphs about databases. I'd ask about a specific script name and get results about scripting in general. The vectors were capturing the vibe but losing the details.

Why hybrid works better

The fix turned out to be surprisingly simple in concept: don't pick one search method — use both.

Vector similarity (cosine distance) handles the semantic, "what does this mean" side. Keyword search (PostgreSQL full-text search) handles the precise, "does this exact term appear" side. At query time, both run in parallel, and the results get combined into a single ranked list.

So now when I search for pgvector, the keyword side immediately surfaces every document that mentions it by name, while the vector side still pulls in semantically related content about vector databases and extensions. Best of both worlds.

The stack at a glance

React + Vite → Go API → PostgreSQL (pgvector + full-text) → Azure AI Foundry Agent

The ingestion pipeline reads Markdown and text files, generates embeddings locally, extracts keywords via Azure OpenAI, and stores everything in Postgres. At query time, the Go backend runs the hybrid search, grabs the best matches, and feeds them as context to the AI agent.

What's next

Now that the text-based chat works, I want to look into voice assistant bots — and explore other ways RAG can be implemented beyond what I've done here. More on that as I go.

Public version of the repo is here.