ASSESSING UNDERGRADUATE RESEARCH EXPERIENCES USING KNOWLEDGE SURVEYS
Our knowledge survey addresses twenty-two knowledge and skill outcomes central to UR. Each item begins with the root “I can” followed by a detailed description of a research-related task (e.g., maintaining field notebooks, reading scientific articles, preparing poster presentations). Students respond to the items using four Likert-scale options that describe their ability to complete the task and their understanding of professional level knowledge. The format and structure of the knowledge survey offers the context of an exam that promotes more informed and intentional self-assessment.
We used the knowledge survey as a pre/post measure in two different research experiences for undergraduates (REU). Prior to UR, rising sophomores (Gateway REU) reported lower mean confidence (1.9 on scale of 1-4) to complete the tasks in the research knowledge and skills sub-construct compared with Advanced REU rising seniors (2.6). After the summer field experience, mean responses rose to 2.8 and 3.2, respectively, with both cohorts reporting significant gains (Cohen’s d = 1.70 and 1.50, respectively). After attending a professional conference, the mean responses increased again (x = 3.1 and 3.3, respectively); the effect sizes for those in the Gateway (Cohen’s d = 1.96) were higher than the Advanced REU (Cohen’s d = 1.64). Control groups, consisting of Year 1-2 and Year 3-4 geoscience majors, had mean responses (2.0 and 2.6, respectively; Cohen’s d = 1.29) similar to the pre-REU Gateway and Advanced cohorts, respectively. These results confirm the value of UR experiences, document the powerful impacts these can have on younger students, and demonstrate the promise of knowledge surveys for assessing undergraduate research.