LESSONS LEARNED FROM THE INTEGRATE PROJECT: THE CHALLENGE OF INTEGRATING SYSTEMS THINKING INTO COLLEGE STEM CURRICULA
To evaluate the impact that these materials had on college students’ systems thinking, a free-response essay item was administered to students at the end of courses where these materials were implemented (treatment group, T, n≈1,200) and at the end of a single large-enrollment course where they were not implemented (control group, C, n=74). All responses were independently scored by at least two raters using the same scoring rubric in which the highest score possible was four. The item and its rubric were developed through a process that drew on the project’s definition of systems thinking and an iterative process of in-class piloting and revision.
A preliminary comparative analysis for a subset of the treatment group’s responses (n=253) and the control group’s responses (n=74) showed that both groups shared the same difficulties in demonstrating systems thinking. In both groups, approximately the same percentage of students: correctly described a natural system and its components (23% T, 20% C); discussed correct interactions between a system’s components (40% T, 42% C); correctly used systems-related concepts, such as feedback and flux (13% T, 12% C); and correctly discussed multiple causal factors (6% T, 5% C). In addition, the same types of misunderstandings and alternate conceptions about systems emerged from the responses from both groups. These preliminary findings suggest that additional research is needed to further understand student learning difficulties associated with systems and to develop more robust curricular interventions that enhance students’ systems thinking skills and dispositions.