CAN CRITICAL INCIDENT QUESTIONNAIRES HELP IMPROVE METACOGNITIVE SKILLS?

We used Stephen Brookfield’s “Critical Incident Questionnaire” (CIQ) as a tool to help students identify and describe class-related experiences that made them aware of the process of their learning. The CIQ asks students to identify when they were most “engaged” and “distanced”, and what incidents they found “helpful”, “confusing”, and “surprising”. Our second-year Environmental Sciences course includes a range of activity types: pre-class preparation, guest speakers, poster sessions, mock town hall meetings, small group discussions, and a field trip. We asked students in this course to respond to the five CIQ questions weekly, and we discussed the responses with the class, also weekly. Our aims here were to determine the frequency and sophistication of metacognitive responses, types of activities that elicited metacognitive responses, and whether individuals’ responses changed systematically over the course of a term. To answer these questions, we coded all CIQ responses using a scheme that differentiates three levels of sophistication.

In general, response sophistication declined over the term. Week 2 produced the greatest proportion of “high-sophistication” responses (25% of total responses, including “other”); in the last three weeks, these dropped below 6%. No individual student showed a clear progression to more sophisticated responses over the term. Guest speakers and poster sessions produced the greatest proportions of “high-sophistication” responses (of responses mentioning those activities). Poster sessions yielded the highest ratio of “engaged” to “distanced” answers (3.4:1), followed by small group discussions and guest speakers (1.6:1).

Our analysis indicates that the CIQ, as implemented in this course, does not produce evidence that students improve their metacognitive skills over a term. It does, however, differentiate more and less engaging/distancing types of activities. This information is useful for future course planning, including future implementation or modification of the CIQ.