THE CONCEPTS IN GEOLOGY ASSESSMENT: AN ALTERNATIVE TO MULTIPLE-CHOICE CONCEPT INVENTORIES

The Concepts in Geology (CiG) assessment was designed as a pre- and post-test for the purpose measuring learning gains, specifically of changes in understanding of the content addressed in a geology class for pre-service teachers. The instructor had previously used a multiple-choice concept inventory, but found it to be very statistically noisy, and not particularly helpful for assessing the effects of curriculum changes.

The CiG consists of ten questions that can be answered with 1-3 phrases or sentences, a single multiple-choice question, and a math question that addresses the numeracy skills needed to understand geologic time. The open-ended questions are scored on three-point scales to allow the assessment of partial understanding. Since test responses are generally hand-written and short-answer, it is currently only useful for small classes.

However, the initial results from 29 paired CiG pre- and post-tests are promising. Cronbach’s alpha, a measure of internal consistency, has a standardized value of 70.0% for the CiG as a pre-test and 75.4% for the post-test. The discrimination coefficient, the correlation between the average score on each individual item and the total test-score, ranged from 9% to 60% for the pre-test, and from 20% to 51% for the post-test. Post-test scores for individual students correlated strongly (r = 70.6%) with their average scores on reading quizzes across the term.

Even with the limited sample size, differences in learning gains were visible within cooperative groups, with the best-prepared students (highest individual reading-quiz average in each group) showing significantly higher normalized learning gains from pre- to post-CiG-test than their team-mates. These students are likely to lead groups, making decisions for and explaining concepts to less-prepared colleagues, which may be the reason they learn more content during the course.