CIG3: AN OPEN-ENDED PRE/POST-TEST FOR INTRODUCTORY GEOLOGY

I developed a pre- & post-test, the CiG3, for a small (up to 24 students) general geoscience class for pre-service teachers. It is made up of ten open-ended questions that can be answered with 1-3 phrases or sentences, one multiple-choice, and one numerical question. The purpose of the open-ended questions is to let me identify partially correct understanding in these students. I have also developed a scoring guide, but scoring each test still takes time, so the test is probably only practical for small classes. The questions address the class objectives (which are based on Ohio’s K-12 science standards) on topics including moon phases, plate tectonics, Earth history, Earth materials, and weather. Primarily, the CiG3 indicates roughly how much students improved their understanding of these topics in the class, but it also answers research questions about individual students or groups within a single class.

The CiG3 is the third version of the pre-post-test piloted in this course so far. 93 students have taken it on the first and last days of class (which lasts 14 weeks). I’m working to determine the reliability of the questions and of the test overall. Although the sample size is small and the questions are diverse and clearly assess multiple constructs (why should a student’s understanding of moon phases be related to their understanding of geologic time?), the CiG3 is somewhat internally consistent (Cronbach’s alpha = 65% on the pre-test, 68% on the post-test; average discrimination coefficient for each question = 46% on the pre-test, 48% on the post-test, with a minimum of 34%). Assessing validity is proving to be more complicated. The post-test scores are moderately (46%) correlated with quiz averages for the 72 students in sections that took quizzes, but the quizzes are multiple-choice, and the distribution of questions about each topic aren’t equally matched. The CiG3 and the quizzes do roughly measure the same thing (understanding of the topics addressed in class). The CiG3 has a wide range of response; so far pre-test scores vary from 0-74% and post-test scores from 21-96%.