TOWARDS DEBUGGING MAPS GENERATED BY GEON APPLICATIONS THROUGH PROVENANCE

Tool Overview

Geoscientists need the capability to understand and debug maps generated from the highly distributed GEON applications and workflows in order to accept them, particularly when a resulting map exhibits unexpected or anomalous properties. On one hand, visualization techniques can help a scientist to understand intermediate and final results of a complex GEON application but not the underlying processes that derived these results. On the other hand, provenance provides information about sources and methods used to derive results, which can also increase the understanding and acceptance of GEON generated maps by scientists. Although rarely used in combination, visualization and provenance techniques together may further increase geoscientists' understanding of GEON maps by providing a complete picture of their generation; scientists would be able to evaluate: final results, derivation processes, and any intermediate result derived during the GEON processes. Probe-It! is a single tool that provides geoscientists with the capability to visualize provenance associated with GEON map generation in order to aid the scientist in understanding and debugging of an unexpected map.

Because GEON maps can be generated using remote resources such as sensory data, remote databases, and services, assessing the quality and correctness of the resultant maps is difficult because of the following: (i) scientists may not know the history associated with some data source; (ii) scientists may not know details about the underlying services applied to their data. Probe-It! addresses those issues by providing visual access to both the sources and processes used to derive a map. For example, through the use of Probe-It!, scientists can move the visualization focus from intermediate and final results of a contour map workflow to the associated provenance trace back and forth. We believe that providing geoscientists with both a description of the map generation process and the means to visualize the associated intermediate and final results will allow scientists to determine whether their resultant maps are correct. Because the most effective visualization varies between scientists, Probe-It! also provides a framework for associating particular visualizations to a particular information type. For example, a GEON scientist may prefer to view spatial datasets as a two-dimensional plot on a map, while another scientist might prefer to view the same data as a graph. Probe-It! is flexible enough to facilitate a multitude of views. 

Figure 1 ProbeIt! snapshot

Figure 1 highlights ProbeIt!'s justification view, which outlines the provenance trail of a contour map workflow that consists of three Web services: dataset retrieval service, a smoothing service, and a contouring service; the resultant map is a contour of gravity data. The provenance associated with this workflow execution includes everything from the specified map region, provided by a geoscientist, to the final contour map of the specified region. The arrows indicate dataflow between services, while each node of the graph represents an invoked Web service and its associated output.

Evaluation Plan

In order to verify that our tool Probe-It! aids scientists in debugging anomalous results, a moderately sized study comprising of gravity experts around the globe is being initiated. Each participant will be asked to complete the following tasks:

1. To identify a correct map. In this task scenario, we will present four gravity contour maps of a region specified by each subject, three of which have had their workflow altered in such a way as to corrupt the final result. The scientists are asked to identify the correct map using only Probe-It! as a resource. If the subjects can both identify the correct map and indicate why the other candidate maps are unsatisfactory, then we can claim that our tool provides both a comprehensive and digest-able trace of a workflow execution. If the subjects fail to identify the correct map or error source, we can at least get an insight on what additional functions might have facilitated success and integrate those missing features into our tool.