WEBWORK ASSESSMENT OF PHYSICAL GEOLOGY LABS: A CASE STUDY

WeBWorK software has been modified and utilized successfully at Wayne State University to produce a cutting-edge Laboratory testing system for >1,000 students who enroll annually in Physical Geology to earn credit for a required science course with a lab. The computerized testing system was conceived and implemented to: (1) eliminate the need to manually enter ~15,000 test scores into spreadsheets; (2) provide each student with a unique test selected from a bank of questions with varying difficulties; (3) enhance the type of questions asked; (4) enhance lab accessibility for special needs students; (5) provide information to reinforce physical samples in some test sets; (6) reduce the opportunity for cheating; (7) provide immediate feedback to answers submitted; (8) eliminate the need to handle large items like topographic maps; and (9) eliminate the need to purchase scantrons.

More than 400 students were tested on a weekly basis during the 2002 Fall semester. Results of a questionnaire answered by >95% of all students indicate that: (1) 54% of all students prefer the WeBWorK system over other testing methods (rated most favorably by freshman and sophomores); (2) immediate feed-back during tests was very unpopular (>70% of students want to eliminate this feature); (3) 75% of all students were happy with images presented on a computer; and (4) the incidence of technology failure or user error was low (<10% and <15% of students indicated any problems during semester).

Implementing the WeBWorK system during the Fall 2002 semester resulted in a 10% reduction in laboratory marks (compared to those obtained in previous years) and a reduction of documented cheating. The decrease in marks was probably caused by both the new WeBWorK testing system providing each student with a unique test (making it more difficult to transfer knowledge about the test) as well as complete revision of all laboratory questions to make them more rigorous. Comparison of assigned (by a panel of instructors and lab assistants) and actual question difficulty indicates that difficulty rankings for some questions need to be recalibrated. The Fall 2002 semester results will be used to redefine the difficulty of questions and thereby modify the question selection process in order to achieve a target average mark of 75% in future labs.