MONITORING UNDERGRADUATE GROWTH THROUGH THE MAJOR USING EMBEDDED ASSESSMENT QUESTIONS

Instructors modify pedagogy or curricula to improve student learning, but valid assessment of the outcomes can be problematic. Nonetheless, measuring changes in student content knowledge and skills across the undergraduate curriculum informs department discussions about program outcomes.

As part of the On the Cutting Edge project, we developed assessments that can be embedded within coursework to monitor student learning related to key concepts across a geoscience program from the introductory through the senior level. Each question set is built around a topic and consists of four questions at increasing Bloom’s Level: question levels 1 and 2 are multiple choice questions that reach Bloom’s levels of comprehension and application; question levels 3 and 4 are short essay questions that reach Bloom’s levels of application, analysis, and synthesis. We developed, created rubrics for, tested and revised fifteen question sets on various topics. In particular, we extensively tested the geologic time question set to show that such embedded assessments can be used to demonstrate growth through the major.

All four geologic time questions were asked at one institution to students in an introductory physical geology course taken by both majors and non-majors, a sophomore-level field geology course taken by majors, and a senior-level research seminar taken by majors. Introductory responses were collected post instruction during the Fall of 2012 (n=131). Sophomore responses were collected during the Fall of 2012 and 2013 (n=73). Senior responses were collected during the Spring of 2013 and 2014 (n=31). Student responses were graded by a single person, and t-tests comparing scores for each question indicate that: 1) The level 1 question successfully (p<0.05) distinguishes sophomore students from introductory students, but does not distinguish sophomore students from senior students; 2) The level 2, level 3, and level 4 questions successfully distinguish sophomore students from introductory students and distinguish senior students from sophomore students.

With ongoing testing of questions and the reliability of the grading rubrics, we expect to create a set of valid and reliable questions within the next year that can be widely used by the geoscience community for program assessment and research needs.