Taxonomic Labeling of Open Ended Feedback: Identifying Cognitive Presence in Survey of STEM Graduate Students

Abstract

Cognitive presence is the extent to which learners are able to construct meaning through discourse and reflection. Evidence of cognitive presence in students indicates the cultivation of a purposeful learning community, a use of critical thinking skills, and a deeper level of understanding of course material. A series of inquiry-based research learning modules was designed to facilitate cognitive presence in STEM graduate students. Following the completion of research learning modules, students responded to a survey that included three free-response questions regarding elements to keep, elements disliked, and elements to add to the course. This study aims to analyze students’ responses to the three free-response questions for evidence of cognitive presence. Student feedback was organized by two student researchers using a hierarchical taxonomy system. Content-focused responses (WHAT categories) used three levels: main categories, categories, and sub-categories. Cognitive presence (WHY categories) used a two-tiered system of main categories and categories. The taxonomy facilitated data labeling of over 300 open-ended student responses. Preliminary analysis of the WHAT categories identified course components students valued. Practical Applications, Topic Exploration, and Peer Review were among the most frequently identified components. WHY categories revealed cognitive presence indicators, with “improving understanding” and “interesting” content being the primary indicators. Future work can include a comprehensive statistical analysis of categorized feedback to better identify patterns across student demographics (online vs. in-person, domestic vs. international) and course components. Statistical analysis will examine the correlations between categories to develop recommendations for optimizing inquiry-based course design.