For science to reliably support new discoveries, its results must be reproducible. This has proven to be a challenge in many fields including fields that rely on computational methods as a means for supporting new discoveries. Reproducibility in these studies is particularly difficult because they require open, documented sharing of data and models and careful control of underlying hardware and software dependencies so that computational procedures executed by the original researcher are portable and can be run on different hardware or software and produce consistent results. Despite recent advances in making scientific work more findable, accessible, interoperable and reusable (FAIR), fundamental questions in the conduct of reproducible computational studies remain: Can published results be repeated in different computing environments? If yes, how similar are they to previous results? Can we further verify and build on the results by using additional data or changing computational methods? Can these changes be automatically and systematically tracked? This presentation will describe our EarthCube project to advance computational reproducibility and make it easier and more efficient for geoscientists to preserve, share, repeat and replicate scientific computations. Our approach is based on Sciunit software developed by prior EarthCube projects which encapsulates application dependencies composed of system binaries, code, data, environment and application provenance so that the resulting computational research object can be shared and re-executed on different platforms. We have deployed Sciunit within the HydroShare JupyterHub platform operated by the Consortium of Universities for the Advancement of Hydrologic Science Inc. (CUAHSI) for the hydrology research community and will present use cases that demonstrate how to preserve, share, repeat and replicate scientific results from the field of hydrologic modeling. While illustrated in the context of hydrology, the methods and tools developed as part of this project have the potential to be extended to other geoscience domains. They also have the potential to inform the reproducibility evaluation process as currently undertaken by journals and publishers.

Geoscientists often spend significant research time identifying, downloading, and refining geospatial data before they can use it for analysis. Exploring interdisciplinary data is even more challenging because it may be difficult to evaluate data quality outside of one’s expertise. QGreenland, a newly funded EarthCube project, is designed to remove these barriers for interdisciplinary Greenland-focused research and analysis via an open data, open platform Greenland GIS tool. QGreenland will combine interdisciplinary data (e.g., glaciology, human health, geopolitics, hydrology, biology, etc.) curated by an international Editorial Board into a unified, all-in-one GIS environment for offline and online use. The package is designed for the open source GIS platform QGIS. QGreenland will include multiple levels of data use: 1) a fully downloadable base package ready for offline use, 2) additional disciplinary and/or high-resolution data extension packages for select download, and 3) online-access-only data to facilitate especially large datasets or updating time series. Software development has begun and we look forward to discussing techniques to create the best open access, reproducible methods for package creation and future sustainability. We also now have a beta version available for experimentation and feedback from interested users and the Editorial Board. The version 1 public release is slated for fall 2020, with two subsequent annual updates. As an interdisciplinary data package, QGreenland is designed to aid collaboration and discovery across fields. Along with discussing QGreenland development, we will also provide an example use case to demonstrate the potential utility of QGreenland for researchers, educators, planners, and communities.

Scientific ocean drilling through the International Ocean Discovery Program (IODP) and its predecessors, has a far-reaching legacy. They have produced vast quantities of marine data, the results of which have revolutionized many geoscience subdisciplines. Meta-analytical studies from these efforts exist for micropaleontology, paleoclimate, and marine sedimentation, and several outstanding resources have curated and made available elements of offshore drilling data, but much of the data remain heterogeneous and dispersed. Each study, therefore, requires reassembling a synthesis of data from numerous sources; a slow, difficult process that limits reproducibility and slows the progress of hypothesis testing and generation. A computer programmatically-accessible repository of scientific ocean drilling data which spans the globe will allow for large-scale marine sedimentary geology and micropaleontologic studies and may help stimulate major advances in these fields. The eODP project, funded through the NSF’s EarthCube program, seeks to facilitate access to and visualization of these large microfossil and stratigraphic datasets. To achieve these goals, eODP will be linking and enhancing three existing database structures: Open Core Data (OCD), the Paleobiology Database (PBDB), and Macrostrat. Over the next three years, eODP will be accomplishing the following goals: (1) enable construction of sediment-grounded and flexible age models in an environment that encompasses the deep-sea and continental records; (2) expand existing lithology and age model construction approaches in this integrated offshore-onshore stratigraphically-focused environment; (3) adapt key microfossil data into the PBDB data model from OCD; (4) develop new API-driven web user interfaces for easily discovering and acquiring data; and (5) establish user working groups for community input and feedback. This project is targeting shipboard drilling-derived data, but the infrastructure will be put in place to allow the addition of other shore-based information. The success of eODP hinges upon interaction, feedback, and contribution of the scientific ocean drilling community, and we invite anyone interested in participating in this project to join the eODP team.

Climate data from Earth System Models are increasingly being used to study the impacts of climate change on a broad range of biogeophysical (forest fires, fisheries, etc.) and human systems (reservoir operations, urban heat waves, etc.). Before this data can be used to study many of these systems, post-processing steps commonly referred to as bias correction and statistical downscaling must be performed. “Bias correction” is used to correct persistent biases in climate model output and “statistical downscaling” is used to increase the spatiotemporal resolution of the model output (i.e. 1 deg to 1/16th deg grid boxes). For our purposes, we’ll refer to both parts as “downscaling”. In the past few decades, the applications community has developed a plethora of downscaling methods. Many of these methods are ad-hoc collections of post processing routines while others target very specific applications. The proliferation of downscaling methods has left the climate applications community with an overwhelming body of research to sort through without much in the form of synthesis guiding method selection or applicability. Motivated by the pressing socio-environmental challenges of climate change – and with the learnings from previous downscaling efforts in mind – we have begun working on a community-centered open framework for climate downscaling: scikit-downscale. We believe that the community will benefit from the presence of a well-designed open source downscaling toolbox with standard interfaces alongside a repository of benchmark data to test and evaluate new and existing downscaling methods. In this notebook, we provide an overview of the scikit-downscale project, detailing how it can be used to downscale a range of surface climate variables such as air temperature and precipitation. We also highlight how scikit-downscale framework is being used to compare existing methods and how it can be extended to support the development of new downscaling methods.

The growing scale of Earth and space science challenges dictate new modes of discovery–discovery that embraces cross-disciplinary interactions and links between communities, between data, between technologies. Nowhere is the challenge more pressing than in the field of Heliophysics where solar energy is generated, propagated through interplanetary space, interacts with the Earth’s space environment, and poses immediate threat to our technological infrastructure and human-natural systems (i.e., space weather). We will present a new project within the National Science Foundation Convergence Accelerator program that represents this new mode of discovery “The Convergence Hub for the Exploration of Space Science (CHESS).” Our approach is to semantically link Heliophysics data through a Knowedge Graph/Network (KG). The presentation and discussion will focus on: - What is a knowledge graph (KG)? - In what ways are KGs poised to transform Earth and space science? - The Convergence Hub for the Exploration of Space Science (CHESS) project and bridging to metadata and knowledge architecture efforts in Heliophysics We will highlight linkages to the NSF EarthCube program and ongoing efforts in the geoinformatics and data science communities across e.g., NSF, NOAA, and NASA.

This repository creates a GUI (graphical user interface) for the BALTO (Brokered Alignment of Long-Tail Observations) project. BALTO is funded by the NSF EarthCube program. The GUI aims to provide a simplified and customizable method for users to access data sets of interest on servers that support the OpenDAP data access protocol. This interactive GUI runs within a Jupyter notebook and uses the Python packages: ipywidgets (for widget controls), ipyleaflet (for interactive maps) and pydap (an OpenDAP client). The Python source code to create the GUI and to process events is in a module called balto_gui.py that must be found in the same directory as this Jupyter notebook. Python source code for visualization of downloaded data is given in a module called balto_plot.py. This GUI consists of mulitiple panels, and supports both a tab-style and an accordion-style, which allows you to switch between GUI panels without scrolling in the notebook. You can run the notebook in a browser window without installing anything on your computer, using something called Binder. Look for the Binder icon below and a link labeled “Launch Binder”. This sets up a server in the cloud that has all the required dependencies and lets you run the notebook on that server. (Sometimes this takes a while, however.) To run this Jupyter notebook without Binder, it is recommended to install Python 3.7 from an Anaconda distribution and to then create a conda environment called balto. Simple instructions for how to create a conda environment and install the software are given in Appendix 1 of version 2 (v2) of the notebook.

The Pangeo project recently introduced large parts of the CMIP6 data archive into the cloud. This enables, for the first time, centralized, reproducible science of state-of-the-art climate simulations without the need to own large storage or a supercomputer as a user. The data itself however, still presents significant challenges for analysis, one of which is applying operations across many models. Two of the major hurdles are different naming/metadata between modeling centers and complex grid layouts, particularly for the ocean components of climate models. Common workflows in the past often included interpolating/remapping desired variables and creating new files, creating organizational burden, and increasing storage requirements. We will demonstrate two Pangeo tools which enable seamless calculation of common operations like vector calculus operators (grad, curl, div) and weighted averages/integrals across a wide range of CMIP6 models directly on the data stored in the cloud. cmip6_preprocessing provides numerous tools to unify naming conventions and parse grid information and metrics (like cell area). This information is used by xgcm to enable finite volume analysis on the native model grids. The combination of both tools facilitates fast analysis while ensuring a reproducible and accurate workflow.

The EarthCube Data Discovery Studio (DDStudio) integrates several technical components into an end-to-end data discovery and exploration system. Beyond supporting dataset search across multiple data sources, it lets geoscientists explore the data using Jupyter notebooks; organize the discovered datasets into thematic collections which can be shared with other users; edit metadata records and contribute metadata describing additional datasets; and examine provenance and validate automated metadata enhancements. DDStudio provides access to 1.67 million metadata records from 40+ geoscience repositories, which are automatically enhanced and exposed via standard interfaces in both ISO-19115 and in schema.org markup; the latter can be used by commercial search engines (Google, Bing) to index DDStudio content. For geoscience end users, DDStudio provides a custom Geoportal-based user interface which enables spatio-temporal, faceted, and full-text search, and provides access to additional functions listed above. Key project accomplishments over the last year include: - User interface improvements, based on design advice from a Science Gateways Community Institute (SGCI) usability team, who conducted user interviews, performed usability testing, and analyzed a dozen of other search portals to identify the most useful features. This work resulted in a streamlined user interface, particularly in presentation of search results and in management of thematic collections. - The earlier effort to publish DDStudio content using schema.org markup resulted in significant usage increase. With over 900K records indexed by Google, nearly half of the roughly 1000 unique users per month are now accessing DDStudio via referrals from Google. - The added ability to harvest and process JSON-LD metadata makes it possible to integrate EarthCube GeoCodes content into DDStudio, and work with this content using DDStudio’s user interface. - New application domains include joint work with the library community, and interoperation with DataMed, a similar system that indexes 2.3 million biomedical datasets.

The ICEBERG (Imagery Cyber-infrastructure and Extensible Building blocks to Enhance Research in the Geosciences) project (NSF 1740595) aims to (1) develop open source image classification tools tailored to high-resolution satellite imagery of the Arctic and Antarctic to be used on HPDC resources, (2) create easy-to-use interfaces to facilitate the development and testing of algorithms for application to specific geoscience requirements, (3) apply these tools through use cases that span the biological, hydrological, and geoscience needs of the polar community, and, (4) transfer these tools to the larger non-polar community.

We present novel applications of yt, a tool originally designed for analysis of astrophysics datasets, to the geosciences. yt is a python-based platform for volumetric data, which enables semantically meaningful analysis and visualization. As part of a wider effort to bring yt to the geosciences, we present an initial use-case of yt applied to 3D seismic tomography models of the upper mantle from the IRIS Earth Model Collaboration. While the rendering capabilities of yt can in general be applied directly to 3D geoscience data, we add several graphical elements to the 3D rendering to aid in interpretation of volume renderings including latitude/longitude overlays and shapefile suport for plotting political and tectonic boundaries along the Earth’s surface. In this notebook, we focus on tomographic models of North America and the Western U.S., where high resolution models and rapidly varying seismic properties provide a rich dataset to explore systematically at a range of lengthscales. The notebook demonstrates loading and rendering of IRIS netcdf models, highlighting interesting 3D features of the Western U.S. upper mantle, and goes on to demonstrate how having direct control of the transfer functions used in creating the final volume rendering allows for a more systematic exploration of the role of the visualization method in our interpretation of 3D volumetric data. Finally, we conclude by demonstrating some of the semantically-aware capabilities of yt for analysis purposes, and demonstrate how these tools have cross-disciplinary functionality.

Ocean waves interact with the environment in many ways. They transport energy and mass, and the resultant sea-surface roughness defines the drag coefficients that transmits wind energy to the ocean (Drennan et al., 2003). Through erosion and deposition, waves change the shape and landscape of coastal areas. Storm surge waves can cause flood damage in coastal areas. Recent studies revealed that wetlands are sensitive to the wave condition, which determines the retreat or growth of coastal ecosystems (Green and Coco, 2007; Mariotti and Fagherazzi 2010). Human activities rely on the condition of waves to conduct marine activities such as fishing, shipping, oil extraction, and offshore constructions. Thus, it is important to understand ocean waves to improve earth system modeling, protect the coastline, predict storm surge, preserve coastal ecosystems, and enhance the offshore business. This project will explore the application of synthetic aperture radar (SAR) imagery to predict significant wave height near the coast. High-frequency (HF) radar data of the ocean (aka CODAR) was used as ground truth data set to calibrate and validate the wave height estimator. Off-shore wind data was also included. The developed code will enhance the current capability to process the satellite data and create a new platform to monitor the coastal environment. The collected data will help further our understanding of the wave spectrum in a coastal environment and the data can support other research in the related topics, e.g. the interaction of waves and ice sheets, wetlands, shorelines, wind farm and aquaculture.

The conduct of reproducible science improves when computations are portable and verifiable. A container provides an isolated environment for running computations and thus is useful for porting applications on new machines. Current container engines, such as Linux Containers (LXC) and Docker, however, have a high learning curve, are resource-intensive, and do not address the entire reproducibility spectrum consisting of portability, repeatability, and replicability. As part of EarthCube, we have developed Sciunit (https://sciunit.run) which encapsulates application dependencies i.e, system binaries, code, data, environment, along with application provenance. The resulting research object can be easily shared and reused amongst collaborators. Sciunit can be used with HydroShare’s JupyterHub CUAHSI notebook environment, and available to the entire community for use. In this poster, we will present three new features in Sciunit which have emerged based on community-provided use cases and discussion. Sciunit is available as a command-line utility. We will: (1) showcase the new Sciunit API. This will allow data facilities to integrate Sciunit as a reproducible environment on portals, (2) show how a Sciunit container can transition to a Docker container and vice versa, and finally, (3) demonstrate the ability to contrast two containers in terms of content and metadata. We will show these capabilities with the Hydrology use case of pySUMMA, a Python API for the Structure for Unifying Multiple Modeling Alternative (SUMMA) hydrologic model.

The GeoSciFramework project (GSF), funded by the NSF Office of Advanced Cyberinfrastructure and NSF EarthCube programs, aims to improve intermediate-to-short term forecasts of catastrophic natural hazard events, allowing researchers to instantly detect when an event has occurred and reveal more suppressed, long-term motions of Earth’s surface at unprecedented spatial and temporal scales. These goals will be accomplished by training machine learning algorithms to recognize patterns across various data signals during geophysical events and deliver scalable, real-time data processing proficiencies for time series generation. The algorithm will employ an advanced convolutional neural network method wherein spatio-temporal analyses are informed both by physics-based models and continuous datasets, including Interferometric Synthetic Aperture Radar (InSAR), seismic, GNSS, tide gauge, and gas-emission data. The project architecture accommodates increasingly large datasets by implementing similar software packages already proven to support internet searches and intelligence gathering. This talk will focus primarily on the Differential InSAR (DInSAR) time-series analysis component, which quantifies line-of-sight (LOS) ground deformation at mm-cm spatial resolution. Here, we compare time series products generated under three different processing techniques. The first, an automated version of InSAR processing using the small baseline subset (SBAS) method performed in parallel on systems such as Generic Mapping Tool SAR (GMT5SAR) and the Generic InSAR Analysis Toolbox (GIAnT). The second method will resemble the first but will implement different processing systems for performance comparison using the InSAR Scientific Computing Environment (ISCE) and the Miami InSAR Time Series Software in Python (MintPy). The final strategy, developed by Drs. Zheng and Zebker from Stanford University, concentrates on the topographic phase component of the SAR signal so that simple cross multiplication returns an observation sequence of interferograms in geographic coordinates [Zebker, 2017]. Our results provide high-resolution views of ground motions and measure LOS deformation over both short and long periods of time.

MagIC (earthref.org/MagIC (https://www2.earthref.org/MagIC)) is an organization dedicated to improving research capacity in the Earth and Ocean sciences by maintaining an open community digital data archive for rock and paleomagnetic data with portals that allow scientists and others to access to archive, search, visualize, download, and combine versioned datasets. A recent focus of MagIC has been to make our data more accessible, discoverable, and interoperable to further this goal. In collaboration with the GeoCodes/P418 group, we have continued to add more schema.org metadata fields to our data sets which allows for more detailed and deep automated searches. We are involved with the Earth Science Information Partners (ESIP) schema.org cluster which is working on extending the schema.org schema to the sciences. MagIC has been focusing on geo- science issues such as standards for describing deep time. We are also collaborating with the European Plate Observing System (EPOS)’s Thematic Core Service Multi-scale laboratories (TCS MSL). MagIC is sending its contributions’ metadata to TCS MSL via DataCite records for representation in the EPOS system. This collaboration should allow European scientists to use MagIC as an official repository for European rock and paleomagnetic data and help prevent the fragmenting of the global paleomagnetic and rock data into many separate data repositories. By having our data well described by an EarthCube supported standard (schema.org/JSON-LD), we will be able to more easily share data with other EarthCube projects in the future.

The project develops innovative tools to extract and analyze the available observational and modeling data in order to enable new physics-based and machine-learning approaches for understanding and predicting solar activity and its influence on the geospace and Earth systems. The heliophysics data are abundant: several terabytes of solar and space observations are obtained every day. Finding the relevant information from numerous spacecraft and ground-based data archives and using it is paramount, and currently a difficult task. The scope of the project is to develop and evaluate data integration tools to meet common data access and discovery needs for two types of Heliophysics data: 1) long-term synoptic activity and variability, and 2) extreme geoeffective solar events caused by solar flares and eruptions. The methodology consists in the development of a data integration infrastructure and access methods capable of 1) automatic search and identification of image patterns and event data records produced by space and ground-based observatories, 2) automatic association of parallel multi-wavelength/multi-instrument database entries with unique patterns or event identifiers, 3) automatic retrieval of such data records and pipeline processing for the purpose of annotating each pattern or event according to a predefined set of physical parameters inferable from complementary data sources, and 4) generation of a pattern or catalog and associated user-friendly graphical interface tools that are capable to provide fast search, quick preview, and automatic data retrieval capabilities. The Team has developed and implemented the Helioportal that provides a synergy of solar flare observations, taking advantage of big datasets from the ground- and space-based instruments, and allows the larger research community to significantly speed up investigations of flare events, perform a broad range of new statistical and case studies, and test and validate theoretical and computational models. The Helioportal accumulates, integrates and presents records of physical descriptors of solar flares, as well as the magnetic characteristic of active regions from various catalogs of observational data from different observatories and heliophysics missions.

Oceanographic research cruises produce abundant data, using a wide range of methods and equipment; very often through large collaborative efforts. These research endeavors span a broad array of disciplines and are critical to investigating the interplay between biological, geological, and chemical processes in the ocean systems over space and time. The advent of genomic sequencing technologies allows for the analysis of gene expression in a variety of environmental settings, to measure the distribution and significance of metabolites and lipids in organisms and the environment. Despite scientists’ best efforts to carefully curate and share their data with collaborators to advance individual studies and publications, no systematic, unifying framework currently exists to integrate ‘omics data with physical, geochemical, and biological datasets commonly used by the broader geoscience community. As a result, the moment each sample leaves the ship is often the last time each data component appears together in a unified collection. Typically, ‘omics datasets are submitted to nucleotide sequence repositories, whereas contextual environmental data are submitted and stored in specialized data-repositories, or only made available within published papers. This makes it difficult to fully reconnect in-situ data, therefore limiting their reuse in other studies. The development of resources to facilitate the aggregation, publication and reuse of biological datasets along with their physicochemical information is critical for studying marine microbes and the biogeochemical processes in the ocean that they drive. We present Planet Microbe, a cyberinfrastructure resource enabling data discovery and open data sharing for historical and on-going oceanographic sequencing efforts. Several historical oceanographic ‘omics datasets (Hawaii Ocean Time-series (HOT), Bermuda Atlantic Time-series (BATS), Global Ocean Sampling Expedition (GOS)) have been integrated into Planet Microbe along with new oceanic large-scale datasets as the Tara Expeditions and Ocean Sampling Day (OSD). In Planet Microbe, these ’omics data have been reintegrated with their in-situ environmental contextual data, including biological and physicochemical measurements, and information about sampling events, and sampling stations. Finally, cruise tracks, protocols and instrumentation are also linked to these datasets to provide the user with a comprehensive view of the metadata. Additionally, Planet Microbe integrates computational tools using National Science Foundation (NSF) funded Cyberinfrastructure (CyVerse) and provides users with free access to large-scale computing power to analyze and explore these datasets.

The most dynamic electromagnetic energy and momentum exchange processes between the upper atmosphere and the magnetosphere take place in the polar ionosphere, as evidenced by the aurora. Accurate specification of the constantly changing conditions of high-latitude ionospheric electrodynamics has been of paramount interest to the geospace science community. In response this community’s need for research tools to combine heterogeneous observational data from distributed arrays of small ground-based instrumentation operated by individual investigators with global geospace data sets, an open-source Python software and associated web-applications for Assimilative Mapping of Geospace Observations (AMGeO) are being developed and deployed (https://amgeo.colorado.edu). AMGeO provides a coherent, simultaneous and inter-hemispheric picture of global ionospheric electrodynamics by optimally combining diverse geospace observational data in a manner consistent with first-principles and with rigorous consideration of the uncertainty associated with each observation. In order to engage the geospace community in the collaborative geospace system science campaigns and a science-driven process of data product validation, AMGeO software is designed to be transparent, expandable, and interoperable with established geospace community data resources and standards. This paper presents an overview of the AMGeO software development and deployment plans as part of a new NSF EarthCube project that has started in September 2019.

The EarthCube BALTO broker (Brokered Alignment of Long-Tail Observations) provides streamlined access to both long-tail and big data using Web Services through several distinct mechanisms. First, we updated the OPeNDAP framework Hyrax, software that serves big data from USGS, NASA, and other sources, with a BALTO extension that tags dataset landing pages with JSON-LD encoding automatically. Therefore, the big data made available through Hyrax are now searchable via EarthCube GeoCODES (formerly P418) and Google Dataset Search. The BALTO broker extension to Hyrax makes thousands of datasets easily searchable and accessible. Second, we focused our efforts on a geodynamics use-case aimed at advancing our understanding of continental rifting processes through the use of an NSF mantle convection code called ASPECT. By addressing this use-case, we implemented a web services brokering capability in ASPECT that allows for remotely accessing datasets via a URL defined in an ASPECT parameter file. Third, through another use-case in ASPECT aimed at testing hypotheses involving global mantle flow, we developed a brokering mechanism for a “plug-in” that accesses NetCDF seismic tomography data from the NSF seismology facility IRIS, then transforms it into the format needed by ASPECT to run global mantle flow models constrained by seismic tomography. Fourth, we demonstrate methods to allow any scientist or citizen scientist to make their in-situ IoT based sensor data collection efforts available to the world. Finally, we are developing a Jupyter Notebook with a GUI that allows for users to search Hyrax servers for big datasets and long-tail data. These cyberinfrastructure developments comprise the entire EarthCube BALTO brokering capabilities.