STALKING THE ELUSIVE LEARNING GAIN IN INTRODUCTORY PHYSICAL GEOLOGY: REAL WORLD COMPLICATIONS

We did a year-over-year comparison of the “learning gains” (LGs) of the physical geology students of 5 faculty who participated in the GARNET (Geoscience Affective Research NETwork) project in 2009-2010 (Y1) and 2010-2011 (Y2). LGs were calculated from pre-instruction (~ 2nd week of term) and post-instruction (~ 14th wk) scores on multiple-choice concept inventories (CIs) modified from the Geologic Concept Inventory of Libarken and Anderson (2005). Budd et al. (2010) showed that GARNET instructors' pedagogical practices correlated with LGs. Here, we hoped to show that changes in instructional methods (“interventions”) adopted by 2 faculty (called N and W) in Y2 resulted in greater LGs than their classrooms showed in Y1.

N's intervention was to add clicker questions to most days' lectures, with responses counting toward course grade, in an effort to boost daily attendance. W added reading reflections for each day's assignment, to be submitted for credit the night before the corresponding lecture.

Two-sample Wilcoxon tests with continuity correction showed large and statistically significant (p < 0.05) increases in median LGs between Y2 and Y1 for N and W. A control group of 3 instructors not doing interventions showed no significant changes (p > 0.05) in median LGs.

Although the interventions appear to have been successful and we endorse them fully, N and W suspect other factors may have significant explanatory power for these results. W's classes met more often in Y2 because state-mandated work furloughs reduced class time in Y1; consequently, Y1 students did not receive the same quality of instruction on certain items covered on the CI. Also in Y2, N joined some other GARNET faculty in administering CIs online instead of on paper. Online administration gives students an unintended opportunity to retain a copy of the CI, possibly resulting in artificial inflation of 14^th week LGs.

Demonstrating learning gains and attributing them to a cause can be elusive.