Some programming languages are easy to develop at the cost of slow execution, while others are lightning fast at run time but are much more difficult to write. Julia is a programming language that aims to be the best of both worlds—a development and production language at the same time. To test Julia’s utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, MPAS-Ocean, as well as a Python shallow water code. Three versions of the Julia shallow water code were created, for: single-core CPU; graphics processing unit (GPU); and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the single-core CPU Julia model was 13 times faster than Python. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20x speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code is extremely fast, with a speed-up of 230-380x compared to the single-core CPU Julia code if communication with the GPU occurs every 10 time steps. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts, and ranges from 2x faster to 2x slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping, but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

A document by Siddhartha Bishnu. Click on the document to view its contents.

This work documents version two of the Department of Energy’s Energy Exascale Earth System Model (E3SM). E3SM version 2 (E3SMv2) is a significant evolution from its predecessor E3SMv1, resulting in a model that is nearly twice as fast and with a simulated climate that is improved in many metrics. We describe the physical climate model in its lower horizontal resolution configuration consisting of 110 km atmosphere, 165 km land, 0.5° river routing model, and an ocean and sea ice with mesh spacing varying between 60 km in the mid-latitudes and 30 km at the equator and poles. The model performance is evaluated by means of a standard set of Coupled Model Intercomparison Project Phase 6 (CMIP6) Diagnosis, Evaluation, and Characterization of Klima (DECK) simulations augmented with historical simulations as well as simulations to evaluate impacts of different forcing agents. The simulated climate is generally realistic, with notable improvements in clouds and precipitation compared to E3SMv1. E3SMv1 suffered from an excessively high equilibrium climate sensitivity (ECS) of 5.3 K. In E3SMv2, ECS is reduced to 4.0 K which is now within the plausible range based on a recent World Climate Research Programme (WCRP) assessment. However, E3SMv2 significantly underestimates the global mean surface temperature in the second half of the historical record. An analysis of single-forcing simulations indicates that correcting the historical temperature bias would require a substantial reduction in the magnitude of the aerosol-related forcing.

This paper presents the first scientific application of local time-stepping (LTS) schemes in the Model for Prediction Across Scales-Ocean (MPAS-O). We use LTS schemes in a single-layer, global ocean model that predicts the storm surge around the eastern coast of the United States during Hurricane Sandy. The variable-resolution meshes used are of unprecedentedly high resolution in MPAS-O, containing cells as small as 125 meters wide in Delaware Bay. It is shown that a particular, third-order LTS scheme (LTS3) produces sea-surface height (SSH) solutions that are of comparable quality to solutions produced by the classical four-stage, fourth-order Runge-Kutta method (RK4) with a uniform time step on the same meshes. Furthermore, LTS3 is up to 35% faster in the best cases, showing that LTS schemes are viable for use in MPAS-O with the added benefit of substantially less computational cost. The results of these performance experiments inform us of the requirements for efficient mesh design for LTS schemes. In particular, we see that for LTS to be efficient on a given mesh, it is important to have enough cells using the coarse time-step relative to those using the fine time-step, typically at least 1:5.

Climate variability and change in the Southern Hemisphere (SH) is influenced by the southern annual mode (SAM) and is closely related to changes in the kinematic properties of the SH surface zonal winds. The SAM and SH surface zonal winds have strong effects on the atmospheric and oceanic circulation system. In this study we investigate the variability and trend in the SAM and position and strength of the surface zonal wind stress (TAUX), using two ensembles of simulations covering the historical record from the Energy Exascale Earth System Model (E3SM-HIST and AMIP) for 1979-2014. In addition, performance of two CO2 forcing simulations from the E3SM (E3SM-1pctCO2 and 4xCO2) is assessed to examine the sensitivity of the variability and changes in the SAM and SH surface TAUX to climate forcing. In general, all E3SM simulations tend to capture the dominant feature of the SAM pattern reasonably well. The annual SAM index in the E3SM-HIST simulation shows a significant increasing trend. These features are similar to the trends in the strength (along with poleward shift in the position) of the annual surface TAUX. For the climatological surface TAUX position and strength, the two CO2 forcing simulations show slightly poleward movement and stronger intensity, while the E3SM-HIST is equatorward and weaker than observations. In the relationship between the SAM and surface TAUX, we show that the SAM index exhibits a positive (negative) relationship with the strength (position) of the surface TAUX in the variability for all seasons and annual mean.

We present a new data assimilation algorithm known as the Continuous Data Assimilation (CDA) algorithm that has been tested extensively in the mathematical literature and, most recently, in a downscaling simulation in the atmospheric literature. Unlike more common data assimilation methods, the CDA algorithm has an exponential convergence rate and is computationally efficient. This work is the first attempt to demonstrate the viability of the data assimilation algorithm in large-scale ocean models. We implement the CDA algorithm in the Model for Prediction Across Scales - Ocean in an idealized mesoscale eddy test case, demonstrating the ability of the data assimilation algorithm to capture the net effects of unresolved processes in low-resolution models.