IS THE CONSISTENCY OF EXPERT-LEVEL TAXONOMIC IDENTIFICATIONS A SIGNIFICANT SOURCE OF ERROR IN BIODIVERSITY AND ECOLOGICAL INVESTIGATIONS?: AN EMPIRICAL ASSESSMENT

When evaluating the results of qualitative identifications the characteristics of human visual and cognition systems that impair experts ability to deliver correct, consistent, and reproducible identifications need to be acknowledged. Yet, this topic has received very little empirical study. To assess how variable expert taxonomic identifications are a set of six mesozooplankton samples from a series of Longhurst Hardy Plankton Recorder net hauls were tally counted by expert zooplankton analysts located at six marine laboratories. The same sample set was assessed on two separate days with over 700 specimens counted and identified on each day. Twenty percent of the analysts returned tally counts that varied by more than ten percent. Thirty-three percent of analysts exhibited low identification consistencies, returning Intraclass Correlation Coefficient scores of less than 0.80. Statistical evaluation of these data suggest that over 83 percent of the observed category count variance can be attributed to inconsistencies in individual analysts’ identifications. We suggest this is the root cause of variation in expert specimen labeling consistency. These results indicate that (1) working with large specimen sets introduces inconsistencies into expert categorizations that can be as large as 300 percent, (2) experts in the taxonomy of organismal groups suffer from the same factors that bias as their less-experienced colleagues, and (3) inconsistency in taxonomic identifications represents a potentially significant, though unacknowledged, source of variation in biodiversity and ecological surveys. Unless organized properly, placed in an environment that minimizes fatigue, given relatively modest amounts of taxonomic identification work to do, provided with time to participate in ongoing training programmes, and encouraged to discuss species concepts and difficult identification situations with comparably trained colleagues high levels of self-consistency, cross-expert consistency, and cross-laboratory consistency are unlikely to be achieved in any taxonomic investigation. The only way we can envision to address this situation is to partner taxonomic experts with automated identification systems and support research into the improvement of this technology.