# Comparative Retrieval Performance: Modules vs Golden Case **Goal**\ Compare CG from current modules with golden case\ \ **Data**\ Modules Kritisches Denken and Agiles Mindset, indexed xapi.\ 20 test questions for each module.\ \ **Method/Approach**\ Retrieval results evaluated using binary relevance score 1-0 (manually labeled relevant chunks).\ \ **Results**\ The best retrieval performance achieved for the module Kritisches Denken was 40% at k3. For Agiles Mindset the best achieved was 50% at k2.\ \ **Evaluation Metrics**\ Retrieval quality: Cumulative Gain (CG at k=1 to k=6)\ \ **Conclusions**\ The Cumulative Gain (CG) performance for the current modules was notably lower than the golden case. Further improvements are necessary to reduce the gap in our retrieval quality when compared to the ideal structured dataset of the golden case. *** ## Content retrieval details After testing with our real courses, we have decided to obtain a golden case of almost 5000 questions and answers and proceed with more experiments Experiments with this golden case and different LLMs do not show a big difference in the retrieved results\ ![](/files/HIhvjhoVP1d6SDUPkabW) But there is a big difference between the results from the golden case (above 90%) and our test learning content (that stays below 40% of retrieved results success) We decided to test different chunking strategies to try to improve the retrieval success scores. After using the sentence splitter strategy we got improved results for our test learning content:
--- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.eduplex.eu/text-analysis/ai-tutor-rag-system/comparative-retrieval-performance-modules-vs-golden-case.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.