THINKING MORE BROADLY AND MORE DEEPLY ABOUT EVALUATION

From two perspectives of both utilizing external evaluation and designing and implementing evaluation, several broad themes have emerged to guide geo- scientists, -educators and -public outreach professionals as they incorporate evaluation into their work. Today, most scientists value the skills and experience that education and outreach professionals bring to a project and incorporate those skills into early planning. However, learning the role and power of evaluation loosely follows the incorporation of broader impacts in research projects during the 1980’s and 1990’s. At that time, scientists routinely requested ‘a line or two of text’ in the final hours of proposal preparation. Evaluation is commonly thought of as an after-thought when, in fact, it is essential to well-planned projects that address the complex (wicked) problems of today.

Evaluation as a social science is evolving. New models are based on development, climate change initiatives, large domestic and international government programs, community-based initiatives in diverse populations, and natural disaster studies. The logic model, one of the models best understood by scientific reviewers, has benefits in organizational thinking but may not be the best model for complex problems.

The value and utility of qualitative data are widely misunderstood. Differing from anecdotal evidence, qualitative data in the form of opinions, examples, critical incidents, and stories can a provide rigorous and in-depth understanding of projects and programs. Likert-scale data, long beloved by people ‘evaluating’ a workshop or program, are rife with assumptions, biases, and superficial analysis.

In both scientific and educational programs, clear articulation of goals is essential to setting up plans to evaluate success. Commonly, activities are thought of as outcomes and goals and, if stated at all, are confused with unmeasurable, long term visions which cannot be measured within the scope of the project. Particularly, the differences of the project’s research and its evaluation are not easy to decipher.

Project management is essential to a successful project. The role of evaluator in monitoring the timeline and activities of a project should not replace the responsibilities of a project’s leader.