RAG Techniques

RAG Techniques

황현지 2026.02.12

Summary

As conversations with AI become commonplace, there's growing interest in technologies that enhance the performance of Large Language Models (LLMs).
Retrieval-Augmented Generation (RAG) is a key technology that significantly improves LLM performance by retrieving relevant information from external databases to generate more accurate responses.
SELF RAG is an improved version of RAG where the LLM determines the need for retrieval and evaluates the accuracy of generated responses on its own. This reduces unnecessary searches and increases the reliability of answers.
Hypothetical Document Embeddings (HyDE) first generates a hypothetical document in response to the user's question and then searches based on this. It's effective in dealing with complex or abstract concepts.
Advanced RAG enhances RAG performance through various techniques such as multi-step retrieval, query expansion, and re-ranking. These techniques help in finding more in-depth and relevant information.
Langcode's CXP (Conversational eXperience Platform) not only applies basic Advanced RAG techniques but also incorporates advanced technologies such as intelligent search type selection, task decomposition, fallback response strategies, and prompt pipelines to provide more accurate and domain-specific answers.
The advancement of these RAG-related technologies is making AI-human interactions more natural and effective, paving the way for more useful and human-centric AI services in the future.

Introduction

Conversing with AI is now becoming a part of everyday life. As shown in the table below, the overall traffic to the website providing ChatGPT services has been consistently trending upward over the past two years, with only slight fluctuations.
similarweb.com
Along with this active usage, methods to use LLMs, which ChatGPT is based on, more accurately and effectively have been proposed. Among these, the Retrieval-Augmented Generation (RAG) technique has been mentioned several times as one of the most effective ways to enhance LLM performance, and is being actively adopted in many LLM-integrated applications.
Briefly, RAG is a method that utilizes information retrieved from external databases in addition to the data learned during LLM development. When a user inputs a request or question, it searches for related documents or data and generates an answer based on that information. This technique provides additional related information for topics that the LLM may not know or is uncertain about, leading to more specific and accurate answers.
However, even RAG cannot overcome all the limitations of LLMs. For instance, there may be cases where incorrect retrieval leads to hallucination issues in the LLM's generated answers, or where the sheer volume of search target documents makes accurate retrieval or comprehensive summarization difficult.
Therefore, RAG technology still faces many challenges, and methods to address these challenges are being researched. In this Insight, we'll look at several methods for improving RAG, as well as how CXP has enhanced traditional RAG.

Main Content

SELF RAG SELF RAG (Self-Reflective Retrieval-Augmented Generation) is an approach that takes RAG a step further. According to research conducted by the University of Washington and IBM team, they enhanced the LLM's ability to judge whether to search or not during the retrieval stage of RAG.
While traditional RAG unconditionally referred to external information or knowledge bases for all user questions, SELF RAG goes through a process of asking and deciding whether additional information is needed to answer the user's question. To do this, it uses "Reflection Tokens" to determine the necessity of retrieval.
Furthermore, SELF RAG goes through a process of evaluating whether the retrieved information is actually helpful and whether its generated answer is accurate. This approach can provide more accurate and reliable answers by reducing unnecessary searches.
Overview of Self RAG
However, to achieve these advantages, additional training of the model is inevitably required to enable the use of Reflection Tokens. A special learning process is needed to judge when to search for information, whether the retrieved information is relevant, and whether the generated content is accurate. The existence of this additional learning stage ultimately diminishes the advantage of traditional RAG, which could reflect real-time changing data without additional learning.
HyDE
HyDE (Hypothetical Document Embeddings) is an approach designed to enhance the retrieval capabilities of RAG systems.
Precise Zero-Shot Dense Retrieval without Relevance Labels
The core idea of HyDE, as introduced in the image above, is to first generate a hypothetical document for the user's question, and then search for related documents based on this. This is similar to how we might imagine a document containing the answer to a question and then look for actual documents that are similar.
Traditional RAG systems use the user's question as-is to search for documents. However, this method sometimes fails to accurately grasp the intent of the question or find highly relevant documents due to linguistic differences between the question and the documents.
HyDE works in the following order:
6
User's question
7
Generate a hypothetical document for the question
8
Embed (vectorize) this hypothetical document
9
Use the embedded hypothetical document to search for similar documents in the actual document database
10
Generate the final answer based on the retrieved documents
This process can better reflect the intent of the question and overcome linguistic differences between questions and documents. It also has the advantage of being applicable without additional learning, just like traditional RAG systems. Moreover, it can be more useful when dealing with relatively complex or abstract concepts.
However, because a hypothetical document is generated for each question, it is disadvantageous in terms of answer generation time and cost. Also, since the quality of the generated hypothetical document directly affects the quality of the search results, if bias or hallucination occurs in the hypothetical document, it negatively impacts the overall quality of the question-answering process. For example, in the case of questions related to complex or specialized domains, the LLM tasked with generation is likely to have no relevant training data, potentially resulting in hypothetical documents with low accuracy.
Advanced RAG
Advanced RAG refers to a collection of approaches that further develop traditional RAG systems. It includes various techniques to improve RAG performance and provide more accurate and relevant information. The main techniques include:
1.Multi-step Retrieval
This technique performs additional searches based on initial search results. For example, it analyzes the content of documents obtained from the first search to extract new keywords or concepts, and uses these for a second search. Through this iterative process, it can find deeper and more relevant information.
2.Query Expansion
This is a technique that expands or reconstructs the user's original question. For example, the question "ethical problems of artificial intelligence" could be expanded to "AI ethics, AI bias, social impact of technology". This allows for the retrieval of more diverse and comprehensive related documents.
3.Re-ranking
This method re-evaluates and ranks the initial search results using a more sophisticated model. For example, language models like BERT, RoBERTa, or T5 can be used separately to more accurately evaluate the relevance between each document and the question. This allows for re-ordering with the most relevant documents to be used in the final answer.
These Advanced RAG techniques can be used independently, but are often combined. For example, multi-step retrieval and query expansion can be used together to expand the query based on the first search results, and then perform a second search using this expanded query.
CXP’s Advanced RAG
Langcode's CXP (Conversational eXperience Platform) has the strength of providing more accurate answers and responding to various enterprise domains. To achieve this, it utilizes not only the aforementioned Advanced RAG techniques but also special techniques.
1.Intelligent Search Type Selection
In RAG, retrieval significantly impacts the quality of the service, so the search method used needs to be approached carefully.
Therefore, CXP analyzes the difficulty and type of user questions and the type of knowledge base to be linked through user question analysis. This analysis allows for intelligent selection of the search type more suitable for the question. Currently, there are at least four types that can be selected, and various types are utilized through single searches and their combinations.
In particular, by leveraging CXP's empirical strengths used in various domains, it can select the most optimal search type for special domain terms or queries.
2.Task Decomposition
The reason for using conversational AI like CXP is to get something done. This means assigning tasks to AI, and depending on how complex the task is, the user may or may not get the desired answer.
CXP evaluates the complexity of the task imposed by the user and decomposes that one task into several sub-tasks. These decomposed tasks use individual Advanced RAG flows, and after strategic aggregation in the final stage, they are organized into a single answer.
At this point, another core technology called the prompt pipeline is linked. Simply put, a prompt pipeline is a technology that generates prompts needed for each task decomposition and execution in detailed units, or combines templates reflecting domain knowledge to ensure that the most appropriate prompt is used for each unit task.
3.Fallback Strategy
In conversational AI chatbots like CXP, the message used when an answer is not possible is generally called a fallback message. In simple RAG-based services, fallback messages typically serve as messages informing that an answer is not possible, rather than generating inaccurate answers when there are no search results or when the content delivered to the LLM is not relevant to the user's question.
However, in CXP, rather than consistently processing with fallback messages, it has a flow that analyzes the cause of the fallback and takes action. For example, it re-delivers to the answer session a flow where the cause has been supplemented, such as whether the problem was in analyzing the user's query, whether a fallback occurred in the decomposed task, or whether the linked knowledge base was inappropriate. In other words, even if it gives up on answering, it looks into whether it was really content to give up on before making a decision.

Conclusion

The development of RAG and related technologies is rapidly evolving as one of the main methods of utilizing LLMs. The progress of these technologies shows great potential not only in conversational AI chatbots but also in various application areas.
In particular, services like Langcode's CXP are actively reviewing and adopting these technological advancements. The aforementioned intelligent search type selection, task decomposition, prompt pipeline, and fallback response, as well as additional technologies such as complex query analysis and knowledge base selection strategies, have been developed not simply by following generally researched content, but by matching the needs of actual clients and the characteristics of various domains.
In conclusion, the development of RAG and related technologies is making AI-human interactions more natural and effective. Through these technological advances, we have been able to implement more useful and human-centric AI services. Langcode will continue to strive for innovative technology development considering customer needs and the characteristics of each domain, and through this, will work to provide a better AI experience.

References

Subscribe to Newsletter (TX Insight)

Langcode publishes a newsletter every month on the third Wednesday of the month (TX Insight).
To stay up to date with Langcode news and the latest IT trends and insights, click the Subscribe button below.