COMPARING REU STUDENT VS. ADVISOR RATINGS OF STUDENT PERFORMANCE INDICATES LOW STUDENT SELF-EFFICACY

Undergraduate researchers and their advisors can have differing opinions of the students’ performance and abilities. We analyze results from performance surveys of 30 School of Ocean and Earth Science and Technology (SOEST) Scholars at the University of Hawaii. The SOEST Scholars program is an academic-year REU that includes research, professional development and mentoring. Each student and advisor completes an identical year-end survey on the student's performance, evaluating "Absolute" performance in 10 areas (e.g., Research Productivity), and "Growth" during the internship in 9 areas (e.g., Works more independently). All survey responses are recorded on a 5-point Likert-style scale (e.g., unsatisfactory to excellent).

For each question, we compare student-advisor responses by quantifying the Likert scale on a scale of 1 to 5, and applying paired, two-tailed t-tests. For the Absolute and Growth datasets as a whole, we use non-parametric permutation tests, which only consider the sign (not the magnitude) of the student-advisor difference, and therefore avoid problems inherent in quantification.

For the "Absolute" questions, the student’s self-assessments consistently underrate their skills and performance relative to their advisor's ratings. Advisor minus student differences (D) range from -0.03 to 0.39 (only 1 of 10 questions had negative D). For all 10 questions combined, the mean student and advisor ratings were 4.06 and 4.24 respectively – a difference that is highly significant (p=0.004). This suggests that mentoring within the SOEST Scholars program, and perhaps in REU's more generally, could be improved by placing additional emphasis on building student confidence and self-appraisal skills, which have been linked to student success and retention.

The "Growth" questions also exhibit some potential differences in student vs. advisor assessment, with the largest differences appearing in questions where advisors underestimated the extent to which students' interest in STEM careers (p=0.08) and graduate school (p=0.06) change over the course of the REU program. This suggests that mentoring and professional development in these areas might be more impactful if they are scheduled towards the end of the REU.