ASSESSING DATA QUALITY WHEN INVOLVING STUDENTS IN RESEARCH: A CASE STUDY USING THE DEVONIAN SEAS STUDENT-SCIENTIST PARTNERSHIP

Assessing the quality of student-generated data in a student-scientist partnership (SSP) is critical to ensure that authentic scientific research is achievable. If data quality is not evaluated student involvement may be more akin to a hands-on lab exercise than an authentic scientific experience and the likelihood of data being used by researchers may be minimal. In development of the Devonian Seas SSP, a project that involves pre-college students in the documentation of Middle Devonian assemblages through time, PRI has focused closely on issues of data quality. Data collected during classroom pilot tests has allowed us to examine the degree of accuracy students of different ages are capable of, the variability of that accuracy, and the types of errors and biases that characterize student-generated data.

Percent accuracy calculated for a given species examined by the students is variable. However, correlation tests comparing data collected by students and researchers are highly statistically-significant for all comparisons. Further analysis of the pilot test data shows that type 1 and type 2 statistical errors are widespread. In general, type 2 errors are prevalent among rare taxa that make up <16% of the sample; for these taxa over 50% of specimens counted may be misidentified by students. Common taxa, or taxa with unique morphologies, do not suffer from the same elevated type 2 errors; 8-16.2% of specimens may be misidentified when examining the most abundant taxon in the sample. Type 2 errors result in an overall smoothing of the relative abundances of different taxa within an assemblage. Preliminary data assessments have resulted in modified research questions with a new focus on lower diversity biofacies in which students examine only the most abundant taxa. Analyses indicate that as long as student errors are randomly distributed, overall assemblage patterns are discernible from student-generated data. Questions of data accuracy, while a great concern when involving students and the general public in research, are not novel to SSPs. Recent studies (Adrain & Westrop, 2000 and Alroy et al, 2001, among others) have focused on errors in paleobiological databases and their potential effects on diversity patterns.