Georgi Cholakov, Asya Stoyanova-Doycheva
University of Plovdiv “Paisii Hilendarski” (Bulgaria)
https://doi.org/10.53656/math2024-3-4-imp
Abstract. The objective of the research is to enhance the functionality of the Evaluator software agent within the Distributed eLearning Center (DeLC), a platform providing extensive support for e-learning activities.
This paper is focused on presenting the latest step in the evolution of one software agent (Evaluator Agent), which helps teachers during evaluating students’ exams, being responsible for evaluation of short free text answers,
provided by the students. To accomplish its job, the agent has its own dictionary, created from a set of words and expressions, provided by test creators for each question of that type, and from the words of each student’s answer,
marked as successful by the teacher. Now this agent’s effectiveness could be increased by extending its dictionary with much larger knowledge base or so-called Large Language Models, made accessible through third party AI,
as ChatGPT. Experiments and results are provided to compare the change in agent’s behavior after the integration with the AI system. The results presented in this work are from first experiments and are deliberately limited
in number to enable manual verification.
Keywords: e-learning, software agents, automated evaluation, LLMs, ChatGPT
