A Multi-Agent RAG Architecture for University Information Management: A Comprehensive Analysis of Performance, Reliability, and Cost

Abstract

University regulations and announcements at Bursa Technical University (BTU) are dispersed across PDF files, web pages, and notice boards, causing information retrieval inefficiencies for students and administrative staff. This study presents the development and quantitative evaluation of BTU-Chatbot, an Agentic Retrieval-Augmented Generation (RAG) powered conversational assistant that consolidates fragmented institutional information into a single citation-aware dialogue interface. Ninety-seven PDF documents were processed using PyPDFLoader and regular-expression-based preprocessing, then embedded into 1536-dimensional vectors using OpenAI’s text-embedding-ada-v2 model and stored in a 29 MB ChromaDB collection. The vector-based retrieval layer selects the three most relevant passages per query using cosine similarity search. The upper layer implements a LangChain-orchestrated multi-agent ReAct loop, in which the retrieve tool accesses the vector database while the Google_search_univ tool performs domain-restricted searches limited to the “*.btu.edu.tr” domain. GPT-4o-mini, with a 128k context window, serves as the generative backbone. System reliability was measured using the RAGAS metric suite. The best performing run achieved Context Recall = 0.97, Context Precision = 0.99, and F1- RP (the F1 score of Context Recall and Precision) = 0.954, demonstrating nearperfect retrieval accuracy. The average cost per query was 6.6×10⁻⁵ USD, with 7.6 s of latency for typical 124-token exchanges. BTU-Chatbot demonstrates that an Agentic RAG pipeline can deliver source-grounded, citation-attributed answers to university-specific queries at low operating cost, although further improvements in generation faithfulness are needed before full deployment.