MEASURING EXPERT PROBLEM-SOLVING IN GEOSCIENCE THROUGH A NATURAL HAZARDS ASSESSMENT

In undergraduate geoscience courses, instructors expect students to acquire not only knowledge but also the ability to think critically and solve complex problems like those they will face as professionals. To effectively teach these skills, instructors need appropriate analytical tools to measure student development of critical thinking and problem-solving skills. To that end, we are developing a method of measuring student problem-solving that reflects the series of decisions that professional scientists across STEM fields make when solving complex problems. Our research uses a natural hazard scenario to measure the differences between expert (professional geologist) and novice (undergraduate student) problem-solving decisions, as well as student expertise development over the course of a natural hazards class.

In the problem, participants are asked to assist the city of Seattle in selecting a section of road to prioritize for structural reinforcement. This assessment focuses on how participants define the most salient features of the problem they are asked to solve, plan out the necessary information they need to solve the problem, interpret and reflect on the information, and select and reflect on their solution – all critical decisions that expert geoscientists must make. We pilot tested the assessment through think-aloud interviews with students and experts, and after confirming that students interpreted the questions as intended, we tested the assessment in undergraduate natural hazards courses last spring. We found that students did not sort critical information from extraneous information in the same way as experts, but the assessment difficulty was not appropriately scaled to detect changes in student problem-solving decision making over the course of one semester. We have modified the assessment in response to these pilot test findings and will be running a full study this coming academic term.