Internal Study of Artificial Intelligence Assistants (AIAs) to Identify the Most Effective AIA For Socratic Learning in Time Series Analysis

Abstract

Socratic-style learning is a form of self-regulated learning (SRL) that uses guided questioning and dialogue to promote deeper understanding. Although effective in traditional settings, facilitating Socratic-style learning in asynchronous environments remains a challenge, as there is a lack of tools capable of replicating Socratic dialogue. While artificial intelligence assistants (AIAs) are becoming increasingly prominent in education, most models are designed to provide direct answers rather than guiding the learner through Socratic dialogue. There is limited research on whether AIAs can support Socratic-style learning, particularly in complex subjects such as time-series analysis (TSA) where solving questions requires a deep conceptual understanding. This study investigates whether conversational AIAs can facilitate Socratic-style learning of TSA. Nine AIAs were tested with a Socratic prompt and ChatGPT-4o, Copilot (1.25063.108.0), and Gemini 2.5 Pro were identified as the only ones capable of following Socratic dialogue. Based on this pre-experiment, a rubric was created to evaluate the AIAs’ step-by-step reasoning, responsiveness, and Socratic engagement. Each identified AIA was given three realistic TSA problems (from undergraduate college courses) and assessed by 3 student researchers, with the scores being averaged across the three questions. From the experiment, Copilot, with an average score of 64.88, demonstrated limited instructional restraint, generated fewer exploratory questions, and loss of continuity with the initial prompt. Gemini 2.5 Pro achieved an average score of 85.19 and ChatGPT-4o achieved an average score of 81.00, demonstrating the strongest abilities to guide users through Socratic questioning. Since Gemini 2.5 Pro and ChatGPT-4o performed similarly, further experimentation may be conducted to identify whether Gemini 2.5 Pro or ChatGPT-4o is better suited for this task.