Optimizing Information Retrieval in RAG Systems via Proactive Hyperparameter Selection

Information retrieval is a crucial operation for Retrieval-
Augmented Generation (RAG) systems, providing substantial gains in
factual accuracy and operational eciency. However, the performance of
a RAG system is highly sensitive to the selection of hyperparameters, optimization of which is computationally expensive and time-consuming. In
this paper, we propose a novel framework, called Proactive Hyperparameter
Estimation and Selection (ProHES), which introduces a paradigm
shift from reactive, brute-force search to an ecient, analytical approach.
The core novelty of the ProHES framework is its two-stage methodology.
First, it analytically estimates the most suitable similarity metric
for a given embedding and dataset by quantifying intrinsic embedding
properties, i.e. sparsity, magnitude meaning, feature independence, and
value spread. Then, it systematically selects the optimal number of retrieved
documents based on user preferences and system requirements.
We evaluate the performance of the ProHES framework across six different
embedding functions on the MS Marco and TriviaQA datasets.
Our results demonstrate that ProHES provides a near-optimal hyperparameter
conguration with 22- to 86-fold reduction in computation
time compared to exhaustive search, achieving an average of 94% of the
best possible performance. The proposed framework represents a significant
advancement by making hyperparameter tuning for RAG systems
more ecient and systematic, thereby paving the way for more accurate,
robust and scalable real-world applications.