Personal tools
 

Retrieval-Augmented Generation (RAG)

Cornell University_011121C
[Cornell University]

- Overview

Retrieval-Augmented Generation (RAG) is an artificial intelligence (AI) technique that improves large language model (LLM) accuracy by pulling relevant facts from an external knowledge base before answering a question. 

RAG is an AI framework that combines the strengths of traditional information retrieval systems - such as search engines and databases - with the capabilities of generative large language models (LLMs). By integrating your data and real-world knowledge with the linguistic skills of an LLM, fact-based generation becomes more accurate, timely, and tailored to your specific needs.

1. How RAG Works:

  • Ingestion: Documents are broken into smaller parts (chunks), converted into numerical representations (vectors), and saved in a database.
  • Retrieval: When you ask a question, the system searches the database for text chunks that match the meaning of your query.
  • Augmentation: The system adds these retrieved text chunks to your original question as context.
  • Generation: The LLM reads the combined prompt and writes an accurate, grounded answer.

 

2. Benefits of RAG:

  • Reduces Hallucinations: Grounds answers in real documents instead of guessing.
  • Saves Money: Avoids the high cost of retraining or fine-tuning models from scratch.
  • Adds Fresh Data: Connects AI models to private or changing company databases in real time. 

 Please refer to the following for more information:

 

- How Does RAG Work? 

Retrieval-Augmented Generation (RAG) works by pulling relevant, factual data from an external knowledge base and feeding it directly into a Large Language Model (LLM) alongside the user's prompt. This allows the AI to reference up-to-date and specific documents - such as company manuals or private databases - without needing to undergo expensive retraining or fine-tuning. 

The entire RAG process operates through a two-phase workflow: the Ingestion Phase (preparing the data) and the Inference Phase (answering the user query). 

1. The Ingestion Phase (Data Preparation)
Before the system can answer questions, external documents must be formatted so an AI search engine can read them quickly: 

  • Chunking: Large files (like 50-page PDFs) are broken down into small, digestible text snippets.
  • Embedding: Each text chunk is converted into mathematical vectors using an embedding model to capture its semantic meaning.
  • Vector Storage: These numerical embeddings are saved into a dedicated vector database (such as ChromaDB or Pinecone).


2. The Inference Phase (Retrieval & Generation) 

When a user asks a question, the dynamic RAG loop executes in real-time: 

  • Query Vectorization: The user's prompt is converted into an embedding using the same mathematical model. 
  • Semantic Search: A retriever queries the vector database using mathematical similarity (like cosine similarity) to find the exact chunks that match the query's meaning.
  • Prompt Augmentation: The integration layer takes the user's original question, combines it with the retrieved text chunks, and crafts an "enriched prompt". 
  • Grounded Generation: The LLM receives this heavily contextualized prompt and generates a response that is directly anchored ("grounded") in the provided factual data, severely reducing the risk of AI hallucinations. 


[More to come ...]

Document Actions