DESIGNING AND EVALUATING A CONCEPTUAL FRAMEWORK FOR GENERATIVE ARTIFICIAL INTELLIGENCE-ASSISTED FEEDBACK: AN EXPERIMENTAL STUDY IN AN UNDERGRADUATE PROGRAMMING COURSE

Date

2025-9-01

Author

Coşkun, Atakan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

166
views

0
downloads

Cite This

Despite the growing integration of artificial intelligence (AI) in education, research on pedagogically grounded AI chatbots remains limited, constraining their instructional potential. To address this gap, a human–AI collaboration feedback model was developed by synthesizing established feedback frameworks. Based on this model, a theory-grounded AI (TGAI) chatbot was designed and compared with a general-purpose AI (GPAI) chatbot. Its impact on student learning was investigated through three pretest–posttest experimental studies in university-level Python and Java programming courses, complemented by a questionnaire and a survey with open-ended responses. Within-group analyses revealed significant improvements for both groups (TGAI and GPAI) in experiment 1. In experiment 2, no significant within-group effects were observed, although the TGAI group approached significance. In experiment 3, only the TGAI group demonstrated a significant gain. Between-group comparisons showed that students interacting with the TGAI chatbot outperformed those using the GPAI chatbot, though this difference was statistically significant only in experiment 2. Questionnaire results indicated no significant group differences. However, qualitative analysis of students’ responses revealed that the TGAI chatbot provided support more closely aligned with pedagogical principles than the GPAI chatbot. In particular, it exhibited a clear advantage in fostering guided discovery learning. Overall, the findings highlight the instructional benefits of aligning AI chatbot design with established feedback principles. The study underscores the potential of the human–AI collaboration feedback model to enhance the educational value of AI-driven tools and recommends further research to validate its effectiveness across diverse learning contexts and disciplines.

Subject Keywords

AI Chatbots, Large Language Models (LLMs), Hybrid Intelligence, Human-AI Collaboration, Human-Computer Interface

URI

https://hdl.handle.net/11511/116150

Collections

Graduate School of Natural and Applied Sciences, Thesis