AN ANALYSIS OF ELECTRONICALLY ARCHIVED STUDENT RESPONSES TO TEXT-ONLY VERSUS ILLUSTRATED CONCEPTUAL QUESTIONS IN LARGE EARTH SCIENCE COURSES

Analyses of 20,738 electronic student responses to text-only and illustrated conceptest questions support the contention that this approach is an inclusive pedagogy to engage students. Students (n=561) used a personal response system (clickers) to answer conceptest questions that assessed student knowledge of topics related to the solid earth during nine, large (n=160), general education, earth science courses taught by two instructors. These questions were categorized as text-only questions (n=45) if they only contained text or illustrated questions (n=113) if they also contained a graphic or hand specimen that required student interpretation to arrive at the correct answer. Repeated measures ANOVAs compared differences in the percentage of correct responses within and between the genders and measures of student prior achievement (ACT-Reading, Math, and Science scores). Instructor was included as a covariate in all analyses because results for both the text-only questions and illustrated questions varied significantly between the nine classes taught.

Within each gender, both men and women correctly answered slightly more illustrated questions than text-only questions (men = 2.6%, women = 0.4%), but these differences were not significant at the p = 0.05 level. Between genders, men correctly answered a slightly higher, but significant, percentage of questions (text-only questions = 3.0%, d = 0.17; illustrated questions = 5.2%, d = 0.39). For prior achievement, there were no significant differences between the percentage of correct responses to text only and illustrated questions for all three ACT tests. Post-hoc comparisons found that only students who scored high on the ACT-Math scores answered a significantly higher percentage of illustrated questions (3%) than text only questions with an effect size of d = 0.21.

These results suggest that including an illustration does not increase the percentage of correct responses for students of all levels of prior achievement. Thus instructors need not be concerned whether their conceptest questions include an illustration, but they should select a conceptest question that best assesses the target concept. Student spatial ability may contribute to student responses on earth science conceptest questions, but more work is needed to establish this connection.