In this presentation, we will describe the [Pangeo Project](http://pangeo.io), a coordinated community effort with support from NASA, NSF, AWS, Microsoft Azure and Google Cloud, to develop interactive and reproducible open source workflows for discovery, visualization, and quantitative analysis of large datasets used for research in the Earth Sciences. The Pangeo computational platform is based on JupyterHub and deployed wherever the data is stored. Python libraries such as Xarray, Rasterio, and Dask enable distributed parallel computations on HPC and Kubernetes clusters. We will discuss the design concepts central to the Pangeo platform and highlight specific applications using NASA satellite data archives on AWS. We will discuss recent progress in the integration of data discovery tools (e.g. STAC, CMR, Intake) with cloud-native storage formats for multidimensional data types (Cloud-Optimized Geotiff, Zarr, etc.) and highlight how they can be used to construct elegant, robust and reproducible scientific workflows. Finally, we will discuss performance, security, transferability across public cloud platforms, cost to operate, and approaches to encourage a cultural shift in scientific computation through educational events.

Many high value data products made available from agencies such as NASA, NOAA, and USGS are broadcast in the clear directly from the satellite. Traditional access patterns involve downlinking the data to an agency ground segment, processing by a science team, and subsequent product distribution through a program such as the NOAA Big Data Initiative or a NASA Distributed Active Archive Center (DAAC). This talk demonstrates the ability to acquire, process, and analyze open broadcast science data entirely in the cloud. Leveraging the new AWS Elastic Ground Station service, we demonstrate selecting and scheduling satellite acquisitions for open science data then leveraging the scalable nature of the cloud to execute a full processing stack. This talk demonstrates the ability to acquire L0 data, leveraging open data processing algorithms from agencies like NASA and NOAA, and a highly scalable, cost effective, and secure processing architecture to produce near real-time data. By taking advantage of direct satellite acquisition and cloud scale processing, we demonstrate reduced latency from acquisition to data availability for scenarios like disaster response, tipping and cueing, and location monitoring. The deployed cloud stack leverages open source components and demonstrates ingest and archival of L0 data, processing to L1 products and above, horizontally scaling data processing algorithms, metadata generation, and search and discovery via APIs and a simple serverless UI.