The geospatial Imaging Spectroscopy Processing Environment on the Cloud (ImgSPEC; formerly GeoSPEC) pioneers an on-demand science data processing system (SDPS) producing user-customized Level 1 calibrated radiance to Level 3+ data products in anticipation for the 2017-2027 Earth Decadal Survey prioritized spaceborne global imaging spectrometer to advance the study of Surface Biology and Geology (SBG). SBG data volumes (~20 TB/day) of high dimensionality (>224 bands) would be infeasible to download and the breadth of applications of the data across dozens of disciplines presents a need to evolve the traditional NASA SDPS. ImgSPEC streamlines processing data into key SBG observables that have demonstrated algorithms at local-to-regional scales and may vary locally. As such, a traditional, monolithic SDPS could not fully exploit the information in SBG measurements. To remove this barrier to use, ImgSPEC demonstrates an on-demand SDPS prototype that improves imaging spectroscopy data discovery, access, and utility enabling shared knowledge transfer from advanced imaging spectroscopy users to less experienced users such as decision makers and the general public. We test three use cases: 1) standard data processing workflows, 2) customized variants of standard workflows, and 3) algorithm development of new workflows. We create collaborative algorithm development environments that offer services typically restricted to NASA SDPSs such as data product provenance and bulk processing. We leverage existing NASA-funded information technologies such as the hybrid on-premise/ cloud science data system (HySDS), the Multi-mission Algorithm and Analysis Platform (MAAP), ECOSIS – a crowd-sourced spectral database, and ECOSML – a crowd-sourced model database. We demonstrate ImgSPEC on the Terrestrial Ecosystem use case processing through to foliar traits and fractional cover, thus aligning with driving thrusts for the SBG Science and Applications Communities.

Before complex analysis of oceanographic or any earth science data can occur, it must be placed in the proper domain of computing and software resources. In the past this was nearly always the scientist’s personal computer or institutional computer servers. The problem with this approach is that it is necessary to bring the data products directly to these compute resources leading to large data transfers and storage requirements especially for high volume satellite or model datasets. In this presentation we will present a new technological solution under development and implementation at the NASA Jet Propulsion Laboratory for conducting oceanographic and related research based on satellite data and other sources. Fundamentally, our approach for satellite resources is to tile (partition) the data inputs into cloud-optimized and computation friendly databases that allow distributed computing resources to perform on demand and server-side computation and data analytics. This technology, known as NEXUS, has already been implemented in several existing NASA data portals to support oceanographic, sea-level, and gravity data time series analysis with capabilities to output time-average maps, correlation maps, Hovmöller plots, climatological averages and more. A further extension of this technology will integrate ocean in situ observations, event-based data discovery (e.g., natural disasters), data quality screening and additional capabilities. This particular activity is an open source project known as the Apache Science Data Analytics Platform (SDAP) (https://sdap.apache.org), and colloquially as OceanWorks, and is funded by the NASA AIST program. It harmonizes data, tools and computational resources for the researcher allowing them to focus on research results and hypothesis testing, and not be concerned with security, data preparation and management. We will present a few oceanographic and interdisciplinary use cases demonstrating the capabilities for characterizing regional sea-level rise, sea surface temperature anomalies, and ocean hurricane responses.