An ICAP-informed rubric for analyzing students' questions to generative AI: engagement patterns and dynamics exploration

Date

2026-07-22

Author

Ding, Lu
Rodenberg, Rachel
Er, Erkan
Nguyen, Ha
Yoon, Meehyun

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

43
views

0
downloads

Cite This

As generative AI (genAI) tools such as ChatGPT become increasingly integrated into education, understanding how students use them to support learning is essential. This study examines undergraduate students' questioning strategies when interacting with ChatGPT to review incorrect answers from a biology exam. Drawing on the ICAP (Interactive, Constructive, Active, Passive) framework, a rubric was developed to assess the cognitive level of student inquiry. A total of 784 student-chatbot interactions were categorized into ICAP modes in order to examine patterns in question types and their association with conversation length, student performance, and the instructional quality of ChatGPT's responses. Findings revealed that most questions were passive or active, whereas constructive and interactive questions emerged more frequently in longer, multi-turn conversations. Higher-performing students posed more constructive questions and exhibited more productive transitions between question types, whereas lower-performing students tended to remain in passive inquiry. ChatGPT's responses were rated highly in accuracy but varied in instructional helpfulness, with longer conversations generally yielding more helpful responses. This study underscores the importance of supporting students in asking generative questions in order to maximize the pedagogical value of genAI tools, and contributes to ongoing conversations about cultivating productive human-AI dialogue. Instructional and design implications are discussed.

URI

https://hdl.handle.net/11511/120245

Collections

Graduate School of Informatics, Article