THE UNDERPINNINGS OF DEVELOPING AN INSTRUMENT TO MEASURE SYSTEMS THINKING ABILITIES IN THE CONTEXT OF EARTH SYSTEM SCIENCE

Earth systems thinking--thinking of the Earth as a complex system made up of interworking subsystems--has been shown to be the highest level of knowing and understanding in the geosciences (Stokes, 2011). Previous work has identified four conceptual frameworks of Earth systems thinking that repeatedly appear in the geoscience education literature (Scherer et al., 2017). These frameworks have also been demonstrated to appear in undergraduate instructor classroom practices (Soltis et al., in press). Based on these frameworks, we have developed an Earth Systems Thinking Concept Inventory (EST CI) to assess undergraduate students’ systems thinking abilities in the context of the Earth system. This study aims to overview the development of the instrument as well as discuss the piloting and validation of this instrument. The instrument was developed using previous qualitative and quantitative data collected from student interviews and a faculty survey. The piloting and validation of the instrument were completed using Amazon’s Mechanical Turk and responses were analyzed using both Classical Test Theory and Item Response Theory. This study will explore both the qualitative and quantitative underpinnings of the development of the EST CI, as well as present initial results about the latent structure of concept inventory items relating to Earth systems thinking and their relationship to frameworks of Earth systems thinking identified in the literature.