Design of a Portable Retrieval-Augmented Generation Architecture for Offline Intelligent Tutoring on Resource-Constrained Hardware
DOI:
https://doi.org/10.18041/1900-3803/entramado.2.13645Keywords:
Artificial Intelligence in Education, Intelligent Tutoring Systems, Retrieval-Augmented Generation, Large Language Models, Educational Technology, Offline Learning Environments, Local Language ModelsAbstract
Cloud infrastructure dependency limits the adoption of intelligent tutoring systems in connectivity-restricted contexts, affecting approximately 2.6 billion people without internet access. Objective: This article presents the design of a portable intelligent tutor that operates fully offline from a USB device on mid-range hardware without a dedicated graphics processing unit. Methodology: The design was approached through Design Science Research (DSR), structured in three iterative cycles covering component selection, architectural integration, and quality evaluator tuning. Results: A reference architecture is proposed that integrates hybrid retrieval (dense vector search, BM25 lexical retrieval, and Reciprocal Rank Fusion), local inference with quantized language models in GGUF format, a pedagogical prompt builder with four teaching modes, an automatic quality evaluator, and AES-256-GCM encryption at rest. The validation, technical and preliminary in scope, consisted of running the system from the same USB drive on three machines with heterogeneous hardware profiles: it operated functionally on a computer with a dedicated GPU (latencies of 54 to 111 seconds per query) and on a mid-range laptop without a GPU (7.7 GB of RAM, 2,048-token context window), while a third machine lacking AVX instructions exhibited an execution failure that delimits the compatible hardware floor. Conclusions: The results suggest a technically viable pathway for reducing access barriers to AI-assisted tutoring in contexts with limited technological resources and connectivity; its pedagogical effectiveness remains to be evaluated with end users.
Downloads
References
1. ABDIN, Marah; ANEJA, Jyoti; BEHL, Harkirat; BUBECK, Sébastien; ELDAN, Ronen; GUNASEKAR, Suriya; HARRISON, Michael; HEWETT, Russell J.; JAVAHERIPI, Mojan; KAUFFMANN, Piero; LEE, James R.; LEE, Yin Tat; LI, Yuanzhi; LIU, Weishung; MENDES, Caio C. T.; NGUYEN, Anh; PRICE, Eric; DE ROSA, Gustavo; SAARIKIVI, Olli; SALIM, Adil; SHAH, Shital; WANG, Xin; WARD, Rachel; WU, Yue; YU, Dingli; ZHANG, Cyril; ZHANG, Yi. Phi-4 technical report [preprint]. 2024. https://doi.org/10.48550/arXiv.2412.08905
2. ANDERSON, Lorin W.; KRATHWOHL, David R. A taxonomy for learning, teaching, and assessing: a revision of Bloom's taxonomy of educational objectives. New York: Longman, 2001. 352 p.
3. BARNETT, Scott; KURNIAWAN, Stefanus; THUDUMU, Srikanth; BRANNELLY, Zach; ABDELRAZEK, Mohamed. Seven failure points when engineering a retrieval augmented generation system. In: Proceedings of the IEEE/ACM International Conference on AI Engineering (CAIN 2024). 2024. p. 194-199. https://doi.org/10.1145/3644815.3644945
4. COGNII. Artificial intelligence for education and training. 2024. https://www.cognii.com/
5. COLOMBIA. CONGRESO DE LA REPÚBLICA. Ley 1581. (17, octubre, 2012). Por la cual se dictan disposiciones generales para la protección de datos personales. Diario Oficial. Bogotá, D.C., 2012. No. 48587.
6. COLOMBIA. PRESIDENCIA DE LA REPÚBLICA. Decreto 1377. (27, junio, 2013). Por el cual se reglamenta parcialmente la Ley 1581 de 2012. Diario Oficial. Bogotá, D.C., 2013. No. 48834.
7. CORMACK, Gordon V.; CLARKE, Charles L. A.; BUETTCHER, Stefan. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In: Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2009). 2009. p. 758-759. https://doi.org/10.1145/1571941.1572114
8. DANE. Indicadores básicos de tenencia y uso de tecnologías de la información y las comunicaciones (TIC) en hogares y personas de 5 y más años de edad 2023. Bogotá D.C.: DANE, 2023. https://www.dane.gov.co/index.php/estadisticas-por-tema/tecnologia-e-innovacion/tecnologias-de-la-informacion-y-las-comunicaciones-tic/indicadores-basicos-de-tic-en-hogares
9. DETTMERS, Tim; PAGNONI, Artidoro; HOLTZMAN, Ari; ZETTLEMOYER, Luke. QLoRA: efficient finetuning of quantized LLMs [preprint]. 2023. https://doi.org/10.48550/arXiv.2305.14314
10. DUOLINGO. Duolingo Max uses OpenAI's GPT-4 for new learning features. 2023. https://blog.duolingo.com/duolingo-max/
11. ECLAC. Universalizing access to digital technologies to address the consequences of COVID-19. Santiago de Chile: United Nations, 2020. https://www.cepal.org/en/publications/45939-universalizing-access-digital-technologies-address-consequences-covid-19
12. FALMAGNE, Jean-Claude; COSYN, Eric; DOIGNON, Jean-Paul; THIÉRY, Nicolas. The assessment of knowledge, in theory and in practice. In: MISSAOUI, Rokia; SCHMID, Jürg (Eds.). Formal concept analysis. Berlin: Springer, 2006. p. 61-79. https://doi.org/10.1007/11671404_4
13. FRANTAR, Elias; ASHKBOOS, Saleh; HOEFLER, Torsten; ALISTARH, Dan. GPTQ: accurate post-training quantization for generative pre-trained transformers [preprint]. 2023. https://doi.org/10.48550/arXiv.2210.17323
14. GAO, Yunfan; XIONG, Yun; GAO, Xinyu; JIA, Kangxiang; PAN, Jinliu; BI, Yuxi; DAI, Yi; SUN, Jiawei; WANG, Meng; WANG, Haofen. Retrieval-augmented generation for large language models: a survey. In: Proceedings of the Conference on AI, Science, Engineering, and Technology (AIxSET 2024). 2024. p. 166-169. https://doi.org/10.1109/AIxSET62544.2024.00030
15. GERGANOV, Georgi. llama.cpp: LLM inference in C/C++ [software]. 2023. https://github.com/ggml-org/llama.cpp
16. GRAESSER, Arthur C.; LU, Shulan; JACKSON, George Tanner; MITCHELL, Heather Hite; VENTURA, Matthew; OLNEY, Andrew; LOUWERSE, Max M. AutoTutor: a tutor with dialogue in natural language. In: Behavior Research Methods, Instruments, & Computers. 2004. vol. 36, no. 2. p. 180-192. https://doi.org/10.3758/BF03195563
17. HEVNER, Alan R.; MARCH, Salvatore T.; PARK, Jinsoo; RAM, Sudha. Design science in information systems research. In: MIS Quarterly. 2004. vol. 28, no. 1. p. 75-105. https://doi.org/10.2307/25148625
18. ITU. Facts and Figures 2023: Internet use. Geneva: International Telecommunication Union, 2023. https://www.itu.int/itu-d/reports/statistics/2023/10/10/ff23-internet-use/
19. KARPUKHIN, Vladimir; OĞUZ, Barlas; MIN, Sewon; LEWIS, Patrick; WU, Ledell; EDUNOV, Sergey; CHEN, Danqi; YIH, Wen-tau. Dense passage retrieval for open-domain question answering. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2020). 2020. p. 6769-6781. https://doi.org/10.18653/v1/2020.emnlp-main.550
20. KASNECI, Enkelejda; SESSLER, Kathrin; KÜCHEMANN, Stefan; BANNERT, Maria; DEMENTIEVA, Daryna; FISCHER, Frank; GASSER, Urs; GROH, Georg; GÜNNEMANN, Stephan; HÜLLERMEIER, Eyke; KRUSCHE, Stephan; KUTYNIOK, Gitta; MICHAELI, Tilman; NERDEL, Claudia; PFEFFER, Jürgen; POQUET, Oleksandra; SAILER, Michael; SCHMIDT, Albrecht; SEIDEL, Tina; STADLER, Matthias; WELLER, Jochen; KUHN, Jochen; KASNECI, Gjergji. ChatGPT for good? On opportunities and challenges of large language models for education. In: Learning and Individual Differences. 2023. vol. 103. p. 102274. https://doi.org/10.1016/j.lindif.2023.102274
21. KHAN ACADEMY. Meet Khanmigo: Khan Academy's AI-powered teaching assistant & tutor. 2024. https://www.khanmigo.ai/
22. LEARNING EQUALITY. Kolibri: the offline app for universal education [software]. 2024. https://learningequality.org/kolibri/
23. LEWIS, Patrick; PEREZ, Ethan; PIKTUS, Aleksandra; PETRONI, Fabio; KARPUKHIN, Vladimir; GOYAL, Naman; KÜTTLER, Heinrich; LEWIS, Mike; YIH, Wen-tau; ROCKTÄSCHEL, Tim; RIEDEL, Sebastian; KIELA, Douwe. Retrieval-augmented generation for knowledge-intensive NLP tasks [preprint]. 2020. https://doi.org/10.48550/arXiv.2005.11401
24. MARTÍNEZ-TORO, Iván. PrivateGPT [software]. 2023. https://github.com/zylon-ai/private-gpt
25. NOGUEIRA, Rodrigo; CHO, Kyunghyun. Passage re-ranking with BERT [preprint]. 2019. https://doi.org/10.48550/arXiv.1901.04085
26. OECD. OECD digital education outlook 2023: towards an effective digital education ecosystem. Paris: OECD Publishing, 2023. 411 p. https://doi.org/10.1787/c74f03de-en
27. OPENAI. Introducing ChatGPT Edu. 2024. https://openai.com/index/introducing-chatgpt-edu/
28. OPENAI. API pricing. 2026. https://openai.com/api/pricing
29. PEFFERS, Ken; TUUNANEN, Tuure; ROTHENBERGER, Marcus A.; CHATTERJEE, Samir. A design science research methodology for information systems research. In: Journal of Management Information Systems. 2007. vol. 24, no. 3. p. 45-77. https://doi.org/10.2753/MIS0742-1222240302
30. PRINSLOO, Paul; SLADE, Sharon. An elephant in the learning analytics room: the obligation to act. In: Proceedings of the International Learning Analytics & Knowledge Conference (LAK 2017). 2017. p. 46-55. https://doi.org/10.1145/3027385.3027426
31. REIMERS, Nils; GUREVYCH, Iryna. Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-IJCNLP 2019). 2019. p. 3982-3992. https://doi.org/10.18653/v1/D19-1410
32. ROBERTSON, Stephen; ZARAGOZA, Hugo. The probabilistic relevance framework: BM25 and beyond. In: Foundations and Trends in Information Retrieval. 2009. vol. 3, no. 4. p. 333-389. https://doi.org/10.1561/1500000019
33. SARTHI, Parth; ABDULLAH, Salman; TULI, Aditi; KHANNA, Shubh; GOLDIE, Anna; MANNING, Christopher D. RAPTOR: recursive abstractive processing for tree-organized retrieval [preprint]. 2024. https://doi.org/10.48550/arXiv.2401.18059
34. WOOLF, Beverly Park. Building intelligent interactive tutors: student-centered strategies for revolutionizing e-learning. Burlington: Morgan Kaufmann, 2009. 480 p. https://doi.org/10.1016/B978-0-12-373594-2.X0001-9
35. WORLD POSSIBLE. RACHEL: Remote Area Community Hotspot for Education and Learning. 2024. https://rachel.worldpossible.org/
36. YAN, Lixiang; SHA, Lele; ZHAO, Linxuan; LI, Yuheng; MARTINEZ-MALDONADO, Roberto; CHEN, Guanliang; LI, Xinyu; JIN, Yueqiao; GAŠEVIĆ, Dragan. Practical and ethical challenges of large language models in education: a systematic scoping review. In: British Journal of Educational Technology. 2024. vol. 55, no. 1. p. 90-112. https://doi.org/10.1111/bjet.13370
37. ZHANG, Peiyuan; ZENG, Guangtao; WANG, Tianduo; LU, Wei. TinyLlama: an open-source small language model [preprint]. 2024. https://doi.org/10.48550/arXiv.2401.02385
Published
Issue
Section
License
Copyright (c) 2026 Entramado
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
