documents flow into a semantic retrieval layer that supplies context to a generative model, backed by an ASP.NET Core API and PostgreSQL with pgvector. Why RAG matters for enterprise knowledge Most organizations already hold the answers to their employees’ questions. They sit in policy documents, wikis, handbooks and runbooks. The hard part is putting the right paragraph in front of the person asking, in the words they used to ask. Three approaches handle this differently: Keyword search matches terms. It works when the user knows the document’s vocabulary, and struggles when they ask about “probation” and the policy says “trial period”. A plain generative API call sends the question straight to a model. The answer reads fluently, but the model only knows its training data and the prompt. It has never seen your leave policy, so it will decline or guess. Retrieval-Augmented Generation (RAG) retrieves relevant passages first and gives them to the model together with the question. The original RAG paper describes combining a model’s parametric memory with a non-parametric memory: a dense vector index reached through a retriever. RAG adds a retrieval step between the question and the model. It can improve access to relevant information; it does not guarantee accuracy. Not every application needs RAG. If answers don’t depend on private or changing documents, a direct model call is simpler. RAG earns its place when answers must come from a specific body of documents. The proof of concept To understand the pipeline end to end, I built a small proof of concept called the Enterprise AI Knowledge Assistant. It uses ASP.NET Core for the API, Google’s Gemini API for embeddings and answer generation, and PostgreSQL with the pgvector extension, hosted on Supabase, as the vector store. The knowledge base is deliberately tiny: a text file of five company-policy statements covering annual leave, remote work, password rotation, probation and deployment approval. That is enough to see every stage of RAG working on questions like “How many days of paid leave do employees get?”. It is a learning project, so this article makes no claims about real users, accuracy or performance. What it does show is the workflow, which is the same whether the knowledge base holds five sentences or five thousand pages. The RAG workflow at a glance Two workflows share one retrieval store. Every RAG system has two phases that meet in a vector store. Phase 1, knowledge preparation (done ahead of time). Read the documents, split them into chunks, turn each chunk into a vector with an embedding model, and store the text, vector and source name. Phase 2, question answering (done per question). Turn the question into a vector with the same model, find the stored chunks closest to it, place those chunks in a prompt, and ask the generative model to answer from them. The rest of this article follows those steps in order. Step 1: Chunking, deciding what the model gets to see Embedding a whole handbook as one vector would blur dozens of topics together, and long passages inflate prompt size and cost. Documents are split into chunks small enough to be about one thing and large enough to make sense alone. The usual choices: Fixed-size splitting is simple and predictable but can cut through sentences. Paragraph-based splitting follows the author’s structure but produces uneven sizes. Overlap repeats a little text across chunk boundaries so a fact that straddles two chunks survives intact in at least one. Metadata, such as the source filename, lets you trace a chunk back to its document. The proof of concept splits on paragraphs and groups whole paragraphs into a chunk until a character limit is reached, keeping each statement intact. The demo file becomes three chunks. Each chunk is stored with its source filename. For longer, messier documents, overlap and token-aware sizing are the natural refinements. Step 2: Embeddings, turning text into vectors An embedding model maps text to a list of numbers so that texts with similar meaning end up near each other. That is what lets “how long is probation?” find “new employees have a probation period of six months” without sharing many words. An embedding does not answer anything. It only helps find candidates, and the nearest vectors can still be irrelevant. The proof of concept calls Gemini’s gemini-embedding-001 model and requests 768-dimensional vectors. According to the Gemini embeddings documentation, this model outputs 3072 dimensions by default and supports smaller sizes such as 768, and the docs also describe retrieval task types (RETRIEVAL_DOCUMENT for stored text, RETRIEVAL_QUERY for questions) that suit RAG. The same page names a newer model, gemini-embedding-2, which is worth evaluating. One rule matters above the rest: documents and questions must be embedded with the same model and dimensionality. Vectors from different models are not comparable. Step 3: Storing and searching vectors in PostgreSQL PostgreSQL with pgvector keeps chunks, metadata and embeddings in one familiar database. Each row in a document_chunks table holds the chunk text, its source file, and its embedding in a vector(768) column. Retrieval is a nearest-neighbor query. In pgvector, <=> is cosine distance, and cosine similarity is 1 - distance according to the pgvector reference. The proof of concept orders chunks by cosine distance to the question vector and keeps the top K: .Select(x => new { x.Id, x.Content, x.SourceFile, Distance = x.Embedding.CosineDistance(queryVector) }) .OrderBy(x => x.Distance) .Take(topK) Choosing K is a trade-off. Too small and the answer may miss a relevant chunk; too large and the prompt fills with noise. The demo uses a small K, which suits its small knowledge base. Without an index, pgvector performs exact nearest-neighbor search by scanning the table, which gives perfect recall (same reference). That is ideal at demo scale. For large collections, Supabase’s vector index guide generally recommends HNSW, and EDB’s pgvector documentation describes it as faster with better recall than IVFFlat, at the cost of memory and build time. That is a decision to make when the data grows, not before. Step 4: Building the augmented prompt and generating the answer This is the step that makes the pipeline RAG. The retrieved chunks are formatted into labelled blocks and placed in the prompt ahead of the question: Context: [Source 1] File: mock.txt Employees are entitled to 12 days of annual paid leave every year. Question: How many days of paid leave do employees get? A system instruction tells Gemini to answer using only the Context, not to fill gaps from general knowledge, and to reply with a fixed “not found” sentence when the context lacks the answer. Compare this with the plain call, where only the question is sent and the model answers from its training data. With retrieval, the model can answer from content it never saw in training, such as a company’s own leave policy. Changing the knowledge means updating the stored chunks, not retraining a model. The API response also returns the retrieved passages with their source file, so a reader can see what the answer was based on. How far can you trust a RAG answer? RAG improves the odds of a relevant answer. It does not guarantee a correct one. Grounding by prompt is an instruction, not a guarantee. A model told to use only the context can still misread it or lean on memory. Retrieval quality sets the ceiling. Top-K returns the closest chunks even when none is truly relevant, which is why production systems often add a similarity threshold or a “no good match” path. Returned sources are not verified citations. They show which passages the model was given, not which passage supports which claim. Evaluating a RAG system means testing retrieval and generation separately: did the right chunk come back, and did the answer stay faithful to it? Taking RAG beyond a proof of concept A few concerns move to the front when real documents and real users arrive: Permission-aware retrieval. Access control has to be applied in the retrieval query, before private content reaches the model. OWASP’s Vector and Embedding Weaknesses entry warns about weak access controls and context leakage between groups in shared vector stores. Untrusted documents. Retrieved text is input, and a document can contain instructions. OWASP’s prompt injection guidance notes that RAG does not fully prevent prompt injection and suggests constraining model behavior and clearly separating untrusted content. More sources and formats. PDFs, Word files, wikis and uploads need text extraction and a way to update or remove documents. Measurement. A set of real questions with known answers, used to tune chunking and K. Where this could help When an organization’s answers live in documents, RAG may reduce the friction of finding them: people ask in their own words and get an answer tied to passages in specific files. Those are potential benefits, not measured results. Building this proof of concept showed me that the model call is the smallest part of the system. Useful AI applications depend on how knowledge is chunked and retrieved, who is allowed to see what, and how answers are checked. How could your team make organizational knowledge easier to reach without sacrificing security or reliability? References Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, arXiv:2005.11401. Google, Embeddings, Gemini API, Google AI for Developers. Supabase, Vector indexes, Supabase Docs. pgvector 0.8.2 reference, pgvector documentation (PDF), hosted by Crunchy Data. EnterpriseDB, Using pgvector, EDB Docs. OWASP, LLM08:2025 Vector and Embedding Weaknesses, OWASP Gen AI Security Project. OWASP, LLM01:2025 Prompt Injection, OWASP Gen AI Security Project.
