BEYOND SPREADSHEETS - R INTERFACE FOR INITIAL ANALYSIS AND VISUALIZATION OF "MEDIUM DATA" FROM ENVIRONMENTAL MONITORING NETWORKS

Earth system monitoring requires multiple sensors and frequent sampling in order to capture spatial and temporal complexities, easily generating hundreds of thousands of data points. Extracting insights from this data is limited by a lack of metadata and difficulty aggregating across data streams or time periods. The data volume size often exceeds spreadsheet capability but is below the threshold of current “big data” initiatives. We present an open-source cloud-based relational database coupled with an R based interface. The tool emphasizes a high level of user-control to facilitate consolidation of data from many different types of nodes, user-defined QA/QC filtering and summary statistics, and easy viewing and querying for the discovery of trends and events. Import/export in CSV is supported, facilitating detailed analysis of sub-sets outside the tool. The tool provides a variety of interactive, publication-quality graphics that can also be fine-tuned by the user. This project maintains an open-access ethos and budget. While developed in support of data management for the 100+ node "Cave Pearl Project" coastal aquifer critical zone observatory, the tool is adaptable to any research effort with data volumes in the hundreds of thousands or millions. Onsite demo will be provided.