Visualization and Eco-hydrologic Models: Opening the black box

Christina Tague
James Frew
Bren School of Environmental Science and Management
University of California, Santa Barbara, 93106, USA
Keywords: Ecohydrology, model, visualization, communication

Abstract

Earth system models synthesize the science of interactions among multiple biophysical and, increasingly, human processes across a wide range of scales. Ecohydrologic models are a subset of earth system models that focus particularly on the complex interactions between ecosystem processes and the storage and flux of water. Ecohydrologic models often focus at scales where direct observations occur: plots, hillslopes, streams, and watersheds, as well as where land and resource management decisions are implemented. These models complement field-based and data-driven science by combining theory and data to create virtual laboratories. Ecohydrologic models are tools that managers can use to ask “what if” questions and domain scientists can use to explore the implications of new theory or measurements. Recent decades have seen substantial advances in ecohydrologic models, building on both new domain science and advances in software engineering and data availability. The increasing sophistication of ecohydrologic models however, presents a barrier to their widespread use and credibility. Because they are “black boxes,” what the models actually do is rarely clear—even to those who design and use them—and this opacity leads to mistrust and complicates the interpretation of model results. For models to effectively advance our understanding of how plants and water interact, we must improve how we visualize not only model outputs, but also the underlying theories that are encoded within the models. In this paper, we outline a framework for increasing the usefulness of ecohydrologic models through better visualization. We outline four complementary approaches, ranging from simple best practices that leverage existing technologies, to ideas that would engage novel software engineering and cutting edge human-computer interface design. Our goal is to open the ecohydrologic model black box in ways that will engage multiple audiences, from novices to model developers, and support learning, new discovery, and environmental problem solving.

Introduction

Earth system models are a broad class of tools that are most commonly used for estimating “what” is likely to happen “if” a given condition occurs. Ecohydrologic models are a subset of earth system models that focus particularly on the complex interactions between ecosystem processes and the storage and flux of water. These models span a wide range of scales, from ecosystem models run at meter scale vegetation plots — to the terrestrial components of global climate models. In this paper, we focus on process or mechanistic ecohydrologic models that encode science-based theory in order to explain how hydrologic and ecologic systems function.
Ecohydrologic models are often used as predictive or scenario generation tools to support environmental management, policy development, and land use or climate change impact assessments. For example, an ecohydrologic model might be used to assess how water quantity and quality may change with changing vegetation management activities such as fuel treatments, restoration, or harvesting. Ecohydrologic models are also used as investigative science tools to enhance understanding of complex eco-hydrologic systems; for example, explaining the potential mechanisms that can lead to a decrease in runoff with forest thinning. Whether they are used for prediction, assessment, or understanding, process-based ecohydrologic models complement purely empirical approaches by providing insight into why observation-based patterns occur. Process-based models (a) often use observational datasets for initialization and parameterization, but extend those observations by explicitly representing mechanisms; (b) investigate both how and why a system might evolve; and (c) can be used for hypothesis testing and to explore the implications of theories. Often these theories arise from intensive field experiments. In this case, mechanistic models serve as virtual laboratories where scientists can explore the implications of field-experiment-based findings across a wider range of conditions
Both of these uses of models—prediction in data-limited contexts and exploring the implications of theory to enhance understanding—require that models encode current scientific theories about how systems work. Models in many ways are libraries of these theories, and models evolve as theories evolve. In ecohydrology, this includes evolving theories about how water is stored and moves through the subsurface. For example, theory in catchment hydrology has evolved from conceptual models of subsurface flow as driven by continuous transmissivity profiles, to conceptual models of hillslopes comprised of areas that connect and disconnect with water conditions (fill and spill approach) and account for macropore flow (Beven & Germann, 2013; Bracken et al., 2013). Similarly, new and sometimes competing theories in ecology are represented in different sub-models of ecohydrologic models. These theories address how plants use water and how they respond to water availability; for example, ecologic theory has considered both hydraulic failure as well as carbon starvation to explain drought-driven tree mortality, and both can be represented in ecohydrologic models (Mencuccini, Manzoni, & Christoffersen, 2019). Similarly, there are competing theories and sub-model representations to address how plants change their allocation of carbon (to stems, leaves, and roots) in response to drought (Franklin et al., 2012).
The mechanistic process-based orientation of ecohydrologic models allows them to be tools for prediction and scenario generation. They can also serve as virtual laboratories and knowledge-based libraries of evolving theories about how ecohydrology works, but this application remains more limited. Recent papers advocate using process-based models for hypothesis testing and behavioral understanding (Clark et al., 2017; Schaefli, Harman, Sivapalan, & Schymanski, 2011); however, this requires that theories embedded in model processes and their interactions be visible to the user. The inherent complexity of ecohydrologic models makes this a key challenge. Indeed, the complexity of modeling tools is often cited as a barrier to their effectiveness for informing decision-making, education, and science-based discovery (Coon, Moulton, & Painter, 2016; Fatichi et al., 2016).

Advancing Visualization: Ways Forward

We propose a range of possible ways to more effectively communicate the embedded knowledge in ecohydrologic models through better visualization of model structure, dynamics, and output. Many of these approaches will be applicable to earth system models in general. We focus here on terrestrial eco-hydrologic models because this domain combines visible features such as forest structure and streamflow with invisible (to the unaided human observer) stores and fluxes such as evapotranspiration and non-structural carbohydrates. Further, much of ecohydrology is done at the plot to small watershed scale, where visualization can be readily tied to human-scale observations: what you can see when you go for a walk. Thus, in ecohydrology we can take advantage of what is familiar to most people in developing model visualizations.
We present four different ways forward. These are not necessarily alternative approaches, but rather options that can be combined in various ways for different audiences, models, and applications. Our proposed options differ in the extent to which they can utilize existing technologies, and in the users and communication goals that they target. Before presenting these different approaches, we discuss the significance of the end user.

The Audience

Consideration of the intended audience is always a central part of designing visualizations. To enhance the credibility and comprehension of ecohydrologic models and their output, we need to account for the level of understanding and the existing conceptual and mental models of the audience (Rapp, 2005). We consider five types of possible users; although there is overlap between each category.
  1. The general public may be relatively unfamiliar with some basic conceptual models of ecohydrology, or even the basics of the water cycle. For this group, ecohydrologic model visualizations may need to build these basic conceptual models and communicate why ecohydrology is interesting or relevant for solving societal problems, and motivate the user to engage with the material.
  2. Students who are actively engaged in learning the science of ecology or hydrology may be more motivated than the general public (even if it is simple motivation to pass a course), but may still need introduction to basic conceptual models, albeit with latitude for additional sophistication.
  3. Field and other domain scientists will have sophisticated understanding of particular components of ecohydrology. A key goal of this group is to use eco-hydrology models to place their domain-specific theory or field research findings into a broader context. For example, consider a field-based study that quantifies plant species differences in drought response by measuring the soil water potential that initiates stomatal closure. The ecohydrology model might facilitate field scientists by estimating the implications of these differences for plant water use given different meteorological forcing conditions or different locations within a landscape. This audience understands ecohydrology but may be unfamiliar with the ecohydrologic model and how it represents the mechanisms that they are interested in and the range of possibilities for model output.
  4. Well-educated managers seek to use the model for scenario development. Here the audience needs to understand how to think about what is included and not included in the model to ensure that it is appropriate for their decision-making context.
  5. Ecohydrologic modelers may want to compare different models, or try to understand why the model produces the patterns that it does. They may be interested in model sensitivity both to parameters and to different sub-model structures.

Approaches for Improving Visualization of Ecohydrologic Models

Our first two approaches focus on model output. Here we assume that by helping users explore and play with model outputs, we can help them to understand what the model does and enhance their learning. Traditionally, investigating model output has occurred within the user interfaces provided by the model’s native environment, or by ingesting model output into generic data analysis software such as Excel, R, MATLAB, etc. While these tools readily support complex data analysis, they do not necessarily help guide learning about the model from its output. Particularly, more novice users, such as the general public and student audiences, may simply not know where to start.
We propose two potential ways to improve the visualization of model output. We then turn to approaches that focus on revealing the structure of the model (and its parameters). Visualization in this case presents the basic assumptions, conceptual models, and ultimately actual equations and parameters that are used in the model. The goal here is to radically change how model documentation is presented and ultimately generated.

Interactive Output Animation

Augmented reality (AR) uses an increasingly available technology to engage users with model output. Augmented reality may be particularly valuable for engaging the public and students. The use of AR for STEM education is a maturing field (see review (Ibáñez & Delgado-Kloos, 2018)) and many review papers focus specifically on earth system and ecological science (e.g. Kamarainen, Reilly, Metcalf, Grotzer, & Dede, 2018; Klippel et al., 2019). Interactive virtual reality, games, and virtual laboratories have shown promise as science education and outreach tools (Castruccio, Genton, & Sun, 2019; Lv et al., 2013; Potkonjak et al., 2016). While evaluating the impact of AR on learning remains an area of ongoing research, there is evidence that the immersive, play-oriented, experiential characteristics of AR tools contribute to inquiry-based learning, and enhance student motivation and spatial abilities (Akçayır & Akçayır, 2017). A key question is whether AR-assisted learning actually leads to the construction of knowledge (rather than simply memory and information).
The innovation of AR is linking “real world” objects with “information about those objects.” Linking environmental model output with observable features in a familiar landscape may help audiences who are mistrustful, overwhelmed, or have other barriers to understanding environmental science that are related to a lack of familiarity. Ecohydrologic model output can be linked to a particular place (e.g. “the” meadow beside a stream), or a particular type of object (e.g. a tree). Mapping model output onto a more “real world” representation allows the user to relate model output to actual landscapes or familiar features within landscapes. At the same time, AR allows the user go beyond what is normally visible. For example, AR could allow users to “see” model estimates of evaporation from an actual tree in a botanical garden.
Recent examples of AR for STEM education occur not only in the classroom but in informal settings: museums, botanical gardens (Ibáñez & Delgado-Kloos, 2018), even shopping malls (BBC’s AR version of Frozen Planet). Virtual learning environments (VLEs) are ARs specifically designed to support public and undergraduate education. A recent example used a VLE to facilitate learning about environmental monitoring data in the Online Watershed Learning System (OWLS) (Smith & Lohani, 2019).
Presenting model output in AR can also be combined with games. Reviews of the use of simulation games and virtual labs for education generally show modest gains in meeting learning objectives, including improved understanding of core science concepts, and non-cognitive or conceptual change, including changing attitudes about science and increased motivation and engagement with learning. For example, Boyle et al. (2016) reviewed literature that documented the effectiveness of games, including simulations, for STEM learning—although results varied with type of game, type of evaluation method, etc. Many of these reviews target K-12, public, and undergraduate education. Recent games, such as WWF’s Free Rivers, Biome Viewer or iBiome-Wetland, and DIY Lake Science, have a hydrologic/ecologic focus, and some include simple models. If we link more sophisticated ecohydrologic models with ARs and VLEs, the biggest gains may be for both upper division undergraduate and graduate students, and field domain scientists.
In summary, recent applications of AR and VLE suggest that games can help the user engage with AR tools and guide them in exploring key relationships. Since ecohydrologic models are frequently used to generate “what if” scenarios, these games can be designed to provide roadmaps to both scenario design and the ecohydrologic mechanisms that lead to different model output under different scenarios. Consider for example a game where the player scores points for determining whether forest thinning impacts of streamflow are increased or decreased under a climate-warming scenario.

Modern Data Mining Techniques for Output Exploration

For more science-savvy audiences (more senior students, researchers from other fields, and modelers themselves), facilitating rapid, structured exploration of model output may be key for using ecohydrologic models for hypothesis development and testing. One of the strengths of ecohydrologic models is their ability to explore multiple interactions between variables, accounting for co-variation across space and time. Ecohydrologic models have been widely used to evaluate core relationships among climate, hydrology and the biosphere. Classic hydrology models explored the relationship between precipitation and streamflow, and vegetation cover and streamflow; ecohydrology models have been used to explore more complex multi-variable interactions, such as between solar radiation (as mediated by aspect), soil water, and plant productivity (Fatichi et al., 2016; Mencuccini et al., 2019).
In many of these papers, model experts selected from hundreds or thousands of different model outputs and even greater numbers of possible relationships in order to develop scientifically interesting conclusions. Experts familiar with model assumptions, parameters, and embedded ecohydrologic theory usually are typically the people who do model output analysis. The experts’ backgrounds allowed them to choose which relationships between model outputs to focus on, and to make strategic choices about appropriate time and space scales. However, for users less familiar with the model (such as field scientists) or students with limited background in ecohydrologic theory, the sheer range of model outputs often limits one’s ability to discover meaningful relationships. In a recent seminar, we presented output from an ecohydrologic model to computer science students. In their application of deep learning techniques to this model output, they made mistakes related to their lack of domain knowledge, such as expecting high correlations between hourly streamflow and precipitation, without accounting for the lag between precipitation and streamflow responses.
Making sense of model output can also be a barrier for domain scientists. In addition to commonly available outputs such as evapotranspiration or net primary productivity, model outputs could also include intermediate variables (such as stem and stomatal conductance.) Many models compute these but do not necessarily output them in order to reduce data volumes. Model output may also be aggregated in space and time in multiple ways (e.g. estimates of hourly evapotranspiration for each point in a grid, versus aggregated annual evapotranspiration over a watershed). Domain experts may be interested in exploring relationships with intermediate or disaggregated variables, but may not even be aware that they are available, particularly for multi-dimensional, complex ecohydrologic models.
Improved user interfaces for exploring model output could be designed to guide both novice and more experienced ecohydrologists in their exploration of model output. A simple approach would be on-screen menus that meaningfully organize core model outputs around particular topics (e.g. precipitation – streamflow, vegetation - streamflow relationships, soil moisture, topographic controls on biogeochemical cycling, etc.). Emerging toolkits such as Shiny (for the R programming environment) allow these types of interfaces to be easily created. More involved approaches for efficient exploration of model output space might use formal semantic indexing and other architectures for organizing information. Studies have shown that these semantic tools can facilitate user searching in geospatial databases (Janowicz & Hitzler, 2017; Jiang et al., 2018). Interfaces that facilitate searching, however, may not help novice users who will simply be overwhelmed by the number and diversity of output options. For these users, an interface that links outputs to conceptual models may be more useful than interfaces designed for searching.
Finding possible outputs is a first step; however, exploring relationships among model outputs is where learning from models occurs. Providing tools for rapidly finding salient relationships in multi-dimensional data may be critical. A wide range of data mining and machine learning techniques are increasingly being used for earth system science data (Bui, 2016; Liu, et al., 2018; Shen, 2018,). Many of these techniques, while developed for observational data, apply equally well to exploring ecohydrology model output. Machine learning techniques that account for temporal lags and can deal with spatial data are particularly relevant (Shen, 2018; Papacharalampous, Tyralis, & Koutsoyiannis, 2019).
Data mining techniques typically require domain knowledge to facilitate selection of appropriate variables (and their time and space scales). While blind application can lead to unexpected discoveries, many argue that domain or expert knowledge is needed to fruitfully apply data mining techniques (Gibert, Horsburgh, Athanasiadis, & Holmes, 2018). When the user is a field scientist or other domain expert, their knowledge of ecohydrologic principles can guide their selection of model outputs. For these users, who may not be familiar with data mining options, visual interfaces could direct them to potential data mining tools and their application to model outputs. For example, an interface could provide a menu of structural equation models that represent some commonly explored relationships between model outputs.
Applying machine learning techniques to model outputs can also help model developers with parameter calibration and evaluation, and with sensitivity and uncertainty analysis. Hybrid approaches that combine the strengths of machine learning for improving the parameterization of physically based approaches have been shown to improve the reliability and accuracy of model predictions (Booker & Woods, 2014; Bui, 2016; Clark et al., 2017). Here again model interfaces could be designed to facilitate the application of machine learning techniques to model output. A key first step is to develop tools that streamline the ingesting of model output into available machine learning software.

Post-hoc visualization of conceptual submodels

For many audiences, particularly domain scientists who know something about the processes being modeled, examining model output may not be enough. These users want to know something about the underlying assumptions, the processes that are included or excluded, the mechanistic detail with which processes are represented, and how these process representations are parameterized. The traditional source for this information has been model documentation, ideally in peer-reviewed journals. Frequently these papers contain cartoon figures that, to varying degrees, capture what the model represents. Often, however, these figures are not detailed enough, providing only a superficial high-level representation of the model. More detailed figures can also be problematic when their level of detail makes them difficult to parse. Many modelers figure out what is in a process-based ecohydrology model by reading the source code. If the code is accessible (e.g. open-source) and is well-structured and well-documented, then this approach can work for an experienced model developer, but not for many domain scientists.
Interactive visualizations of conceptual hierarchical models might offer a more user-friendly way to communicate underlying model structures. Similar visualization techniques have already been used to support the coupling of disparate models (or submodels) within flexible modeling systems (Coon et al., 2016). In addition to supporting the software engineering task of coupling different models, these visual tools can be used post-hoc to support model documentation. These types of approaches for visualizing model structure are particularly valuable when there is consistency in how model structure is described. For example, CDMS, a model coupling system, provides a GUI to support model coupling (Peckham, Hutton, & Norris, 2013). As part of this tool, model developers provide model descriptions (as an HTML help document) in a consistent format that is easily accessible as part of the coupling framework. These documents are standardized to include an extended model description, references, the main equations of the model, sample input and output, and acknowledgment of the model developer(s). Similarly, model development frameworks that use tools like dependency graphs are examples of how these structured descriptions of models can be generated (Coon et al., 2016). These model description tools are often text-based, as opposed to graphical, but they are a step in the right direction and could be expanded upon to provide visual representations.
Ultimately the presentation of model structural information needs to address multiple audiences. The complexity of underlying model structures means that a single conceptual figure is unlikely to satisfy the needs of diverse audiences, or even a single user who seeks to understand multiple components of the model. To address these, visual representations will need to be hierarchical, so users can explore details as needed. Hierarchical structures also reflect the underlying best practices in software engineering that build complexity through modularity.
The hydrologic community has already had some success with improving metadata and documentation standards in order to improve data accessibility and usability. Hutton et al. (2016) argue that a similar effort to define metadata standards for model documentation is needed. We concur but also emphasize that a visual hierarchical approach that can support different audiences is needed. Community databases for sharing models and data such as Hydroshare (Horsburgh et al., 2016) provide platforms and metadata standards that contribute to sharing. Improved visualization tools would complement and extend these efforts.

Automating visual Representation of model structure

The previous section makes the case for visual representations of model structure within a user interface that facilitates exploration from the diverse perspective of different audience types—from a public that may benefit from a pictorial representation of a carbon cycle, to a field scientist who may want to know the details of the submodel used to estimate photosynthesis.
However, such visual representations can be difficult to create and deploy in conjunction with existing models. Incremental development by generations of researchers more trained in science than software engineering can lead to models whose code is difficult to understand. Model documentation to support replication of research results is increasingly required for peer-reviewed publications, but this may be limited to documenting workflows, data, and model structures, rather than multiple visual representations to support understanding of model outputs and/or structure.
The challenge in this case is to reduce the time and effort required to visualize model structure. Tools that at least semi-automate model documentation are increasingly available and recommended for earth system and biological science model development (Karimzadeh & Hoffman, 2018). Adding visual elements to code that could then be used to generate visualizations on the fly might be a next step. Efforts to standardize metadata for model description move in this direction (Gil, Ratnakar, & Garijo, 2015), but they do not necessarily provide easy access to model structural components, nor are they searchable and visual.
Graphical/visual programming environments, or model building tools that translate visual elements into code, have been used for decades to help students learn to develop models (STELLA is a well-known example, but there are many others (Navarro-Prieto & Cañas, 2001), as well as domain specific visual programming environments tailored to hydrology (e.g. GeoVISTA (Takatsuka & Gahegan, 2002)) and biology (e.g. BioUML (Kolpakov, Puzanov, & Koshukov, 2006)). While these tools have strengths within an educational context, more complex ecohydrology and earth system science models do not readily fit into these environments—because they require computational efficiency, code architectures, and memory resources that these tools generally do not support.
In their relatively recent review of virtual worlds, Potkonjak et al. (2016) did not find examples where the complex system dynamics of individual components (e.g. a complex model of photosynthesis within an ecohydrology model) were incorporated into these worlds. In other words, there remains a gap between highly visual model interfaces for education, and the complex ecohydrology models that support developing new science and science applications. Addressing this gap may require a community, similar to the multi-institution large-group development of virtual laboratories, that support shared innovation in areas such as physics (Potkonjak et al., 2016).
What is needed is essentially a reverse engineering of these approaches, where visual or graphical elements are linked with the text of the source code comprising ecohydrologic models. New advances in visualization languages and tools that provide such graphical notation may be a way forward (Holm-Peterson et al., 2014). The ideal interface would be multi-level, adaptable, searchable, and provide information on inputs, outputs, model structure, and submodels.

What’s next

In this section we propose three areas where we believe progress is most needed and can beneficially advance by “opening the black box” of ecohydrologic and earth system models.

Software engineering

Software engineering best practices applied to current and future models can go a long way towards facilitating visibility into model structure and operation. In addition to the obvious benefits of readable code (the software is its own documentation) and component reuse (portability of the human reader’s knowledge), there are two structural benefits that can specifically impact model visualization:
  1. Component graph: To the extent that the model’s software modules directly reflect the model’s conceptual components, the flow of data and control during the model’s execution can be automatically constructed from a combination of function call graphs (static) and execution profiling (dynamic). The resulting graph can serve as a scaffold for model visualization, as well as important documentation in its own right. This strongly argues for extra engineering effort to preserve the structural relationship between the model’s conceptual and software components.
  2. Rendering hints: Given a component graph, it is straightforward to visualize a model as “boxes and arrows”; i.e., as component processes and the flows of data and control between them. It is more challenging to visualize the operation of components or data using visual metaphors appropriate for the specific content. We suspect that semantic tagging of model components and data will prove useful here. For example, a submodel representing photosynthesis in conifers could be tagged such that the light wavelengths and leaf structures involved could be rendered by a generic visualization environment that had no specific knowledge of the biophysics of photosynthesis or vegetation.
In both of these cases the goal is to automate to maximum extent possible the generation of visualizations of model structures and operation.

Visualization effectiveness

Noteworthy in reviews of the effectiveness of AR and gaming or STEM education are calls for additional “metacognitive scaffolding and experimental support” (Ibáñez & Delgado-Kloos, 2018). In other words, users (especially novice ones) often need guidance in the use of such tools. Guidance may be part of the interface, such as games with increasing levels of complexity. These recommendations are likely to be applicable in the design of more sophisticated tools for using model output to understand the underlying science in the model.
Critiques of AR frequently emphasize issues with navigation and usability; specifically, the problem of dealing with “too much information.” Similarly, reviews of virtual laboratories and games for science education often find that the complexity of the interface is a barrier to its ease of use and effectiveness (Boyle et al., 2016; Potkonjak et al., 2016). Not surprisingly “Ease of use” and “perceived usefulness” are commonly cited attributes for the effectiveness of games and other technology for STEM education (Šumak, Hericko, & Pušnik, 2011). Further, some studies comment on the challenge of differentiating between “games” as entertainment and games for more serious learning (Boyle et al., 2016).
Reviews also consistently advocate for more evaluation, and it is clear that more work is needed in designing games and visual interfaces in general to best achieve objectives, whether these are knowledge acquisition, or behavioral, attitudinal, or motivational change. Success in interface design for improving the use of ecohydrologic models will depend on how well tools are designed, and on using iterative feedback to evolve the design. Formal techniques for design and assessment can be used for this evaluation (e.g. Technology Acceptance Model (Šumak et al., 2011), User Engagement (O’Brien & Toms, 2008), ARCS (attention, relevance, confidence, satisfaction) design principles for effective learning (Keller, 2008), and measures that formally test knowledge acquisition).

Community support

In this paper, we argue that to make more effective use of ecohydrologic models for science discovery, education, and environmental problem solving will require transforming how we visualize not only model results, but also the knowledge embedded in these digital laboratories.
In our review we have identified several promising directions that include new technologies and new approaches, and we acknowledge that there are likely many examples that we have missed. At this time, these technologies and practices are not widely used within the ecohydrologic modeling community and further existing tools simply do not go far enough. We need new strategies that can be tailored to address multiple visualizations objectives and multiple audiences.
To do more, however, will require the engagement of the ecohydrologic community and beyond. This engagement needs to be supported on multiple fronts. Improving model visualization requires time and effort, so there must be incentives to support this. Within the academic and science funding communities, there are increasing efforts to provide credit for software products as well as publications, and to support new cyberinfrastructure (Howison and Bullard, 2015, Stall et al., 2018). This is a step in the right direction. But significant advances will require funding that supports extended interdisciplinary collaborations such as working groups and centers. It is only through interdisciplinary collaboration among model developers and users, field data scientists, communication experts, and computer scientists that we can build a new generation of environmental model visualizations. Without engaging experts in those fields, ecohydrologic modelers will miss the rich and evolving body of work on the technologies, their best practices, and perhaps most importantly—the art of effective visualization design for communication and learning. In the end “opening the black box” of environmental models will require opening the box of how we architect environmental models and present their output. This will take real work, but results could greatly accelerate the contributions of ecohydrologic models for evolving science and bringing science findings to the many communities that can benefit from this knowledge.

Acknowledgements

NSF-SERI fire (Grant 1520847); Crossroads program at UCSB, and members of the Crossroads working group (Hollerer, Wendy Meiring, and graduate students in the Opening the Black Box of Environmental Models seminar); Janet Choate for her help with figures and editing. Ethan Turpin for insights into model visualization from an artistic perspective.
*Data sharing not applicable to this article as no datasets were generated or analysed

References

Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature.Educational Research Review, 20 , 1–11. https://doi.org/10.1016/j.edurev.2016.11.002
Beven, K., & Germann, P. (2013). Macropores and water flow in soils revisited. Water Resources Research,  49 (6), 3071-3092. https://doi.org/10.1002/wrcr.20156
Booker, D. J., & Woods, R. A. (2014). Comparing and combining physically-based and empirically-based approaches for estimating the hydrology of ungauged catchments. Journal of Hydrology, 508 : 227–39. https://doi.org/10.1016/j.jhydrol.2013.11.007
Boyle, E. A., Hainey, T., Connolly, T. M., Gray, G., Earp, J., Ott, M., … Pereira, J. (2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education, 94 , 178–92. https://doi.org/10.1016/j.compedu.2015.11.003
Bracken, L. J., Wainwright, J., Ali, G. A., Tetzlaff, D., Smith, M. W., Reaney, S. M., & Roy, A. G. (2013). Concepts of hydrological connectivity: Research approaches, pathways and future agendas. Earth-Science Reviews119 , 17-34. https://doi.org/10.1016/j.earscirev.2013.02.001
Bui, E. N. (2016). Data-driven critical zone science: A new paradigm.Science of the Total Environment, 568 , 587–93. https://doi.org/10.1016/j.scitotenv.2016.01.202
Castruccio, S., Genton, M. G., & Sun, Y. (2019). Visualizing spatiotemporal models with virtual reality: From fully immersive environments to applications in stereoscopic view. Journal of the Royal Statistical Society: Series A (Statistics in Society), 182 (2), 379-387. https://doi.org/10.1111/rssa.12381
Clark, M. P., Bierkens, M. F. P., Samaniego, L., Woods, R. A., Uijlenhoet, R., Bennett, K. E., … Peters-Lidard, C. D. (2017). The evolution of process-based hydrologic models: Historical challenges and the collective quest for physical realism. Hydrology and Earth System Sciences, 21 , 3427-3440. https://doi.org/10.5194/hess-21-3427-2017
Coon, E. T., Moulton J. D., & Painter, S. L. (2016). Managing complexity in simulations of land surface and near-surface processes. Environmental Modelling & Software, 78 , 134–49. https://doi.org/10.1016/j.envsoft.2015.12.017
Fatichi, S., Vivoni, E. R., Ogden, F. L., Ivanov, V. Y., Mirus, B., Gochis, D., … Tarboton, D. (2016). An overview of current applications, challenges, and future trends in distributed process-based models in hydrology. Journal of Hydrology, 537 , 45–60. https://doi.org/10.1016/j.jhydrol.2016.03.026
Franklin, O., Johansson, J., Dewar, R. C., Dieckmann, U., McMurtrie, R. E., Brannstrom, A., & Dybzinski, R. (2012). Modeling carbon allocation in trees: a search for principles. Tree Physiology , 32 (6), 648–666. http://dx.doi.org/10.1093/treephys/tpr138
Gibert, K., Horsburgh, J. S., Athanasiadis, I. N., & Holmes, G. (2018). Environmental data science.Environmental Modelling & Software, 106 , 4–12. https://doi.org/10.1016/j.envsoft.2018.04.005
Gil, Y., Ratnakar, V., & Garijo, D. (2015). OntoSoft: Capturing scientific software metadata. InProceedings of the 8th ACM International Conference on Knowledge Capture (K-CAP) , Palisades, NY, 32 , 1-4. https://doi.org/10.1145/2815833.2816955
Holm-Peterson, M., Abeln, C. O., Casey, T. M., Chan, S. Y., Chronister, B. L., Dunmore, I. M., … Weber, J. (2014). Declarative approach for visualization. Google Patent US 8,810,595 B2.
Horsburgh, J. S, Morsy, M. M., Castronova, A. M., Goodall, J. L., Gan, T., Yi, H., … Tarboton, D. G. (2016). Hydroshare: Sharing diverse environmental data types and models as social objects with application to the hydrology domain. JAWRA Journal of the American Water Resources Association, 52 (4), 873–89. https://doi.org/10.1111/1752-1688.12363
Howison, J. and Bullard, J. (2016), Software in the scientific literature: Problems with seeing, finding, and using software mentioned in the biology literature. J Assn Inf Sci Tec , 67: 2137-2155. doi:10.1002/asi.23538
Hutton, C., Wagener, T., Freer, J., Han, D., Duffy, C., & Arheimer, B. (2016). Most computational hydrology is not reproducible, so is it really science? Water Resources Research, 52 (10), 7548–55. https://doi.org/10.1002/2016WR019285
Ibáñez, M. B., & Delgado-Kloos, C. (2018). Augmented reality for stem learning: A systematic review. Computers & Education, 123 : 109–23. https://doi.org/10.1016/j.compedu.2018.05.002
Janowicz, K. & Hitzler, P. (2017). Geospatial semantic web. In International Encyclopedia of Geography: People, the Earth, Environment and Technology, Eds. Richardson, D., Castree, N., Goodchild, M. F., Kobayashi, A., Liu, W., & Marston, R.A. https://doi.org/10.1002/9781118786352.wbieg1158
Jiang, Y., Li, Y., Yang, C., Hu, F., Armstrong, E. M., Huang, T., … & Finch, C. J. (2018). Towards intelligent geospatial data discovery: A machine learning framework for search ranking.International Journal of Digital Earth, 11 (9), 956–971. https://doi.org/10.1080/17538947.2017.1371255
Kamarainen, A., Reilly, J., Metcalf, S., Grotzer, T., & Dede, C. (2018). Using mobile location-based augmented reality to support outdoor learning in undergraduate ecology and environmental science courses. Bulletin of the Ecological Society of America, 99 (2), 259–276. doi:10.1002/bes2.1396.
Karimzadeh, M., Hoffman, M. M. (2018) Top considerations for creating bioinformatics software documentation, Briefings in Bioinformatics , 19 (4), 693-699. https://doi.org/10.1093/bib/bbw134
Keller, J. M. (2008). First principles of motivation to learn and e3 - learning.” Distance Education,29 (2), 175–185. https://doi.org/10.1080/01587910802154970
Klippel, A., Zhao, J., Jackson, K. L., La Femina, P., Stubbs, C., Wetzel, R., … Oprean, D. (2019). Transforming earth science education through immersive experiences: Delivering on a long held promise. Journal of Educational Computing Research, 57 (7), 1745–1771. https://doi.org/10.1177/0735633119854025
Kolpakov, F., Puzanov, M., & Koshukov, A. (2006). BioUML: Visual modeling, automated code generation and simulation of biological systems. In Proceedings of the Fifth International Conference on Bioinformatics of Genome Regulation and Structure , 281–84.
Liu, Z., Peng, C., Work, T., Candau, J. N., DesRochers, A., & Kneeshaw, D. (2018). Application of machine-learning methods in forest ecology: recent progress and future challenges. Environmental Reviews26 (4), 339-350.
Lv, Z., Tek, A., Da Silva, F., Empereur-mot, C., Chavent, M., Baaden, M. (2013) Game on, science - how video game technology may help biologists tackle visualization challenges. PLoS ONE, 8 (3), e57990. https://doi.org/10.1371/journal.pone.0057990
Mencuccini, M., Manzoni, S., & Christoffersen, B. (2019). Modelling water fluxes in plants: from tissues to biosphere. New Phytologist , 222 , 1207-1222. https://doi.org/doi:10.1111/nph.15681
Navarro-Prieto, R., & Cañas, J. J. (2001). Are visual programming languages better? The role of imagery in program comprehension. International Journal of Human-Computer Studies , 54 (6), 799–829. https://doi.org/10.1006/ijhc.2000.0465
O’Brien, H. L., & Toms, E. G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology, 59 (6), 938–955. https://doi.org/10.1002/asi.20801
Papacharalampous, G., Tyralis, H., & Koutsoyiannis, D. (2019). Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes. Stochastic Environmental Research and Risk Assessment,  33 481–514. https://doi.org/10.1007/s00477-018-1638-6
Peckham, S. D., Hutton, E. W. H., & Norris, B. (2013). A component-based approach to integrated modeling in the geosciences: The design of CSDMS. Computers & Geosciences, 53 , 3-12. https://doi.org/https://doi.org/10.1016/j.cageo.2012.04.002
Potkonjak, V., Gardner, M., Callaghan, V., Mattila, P., Guetl, C., Petrović, V. M., & Jovanović, K. (2016). Virtual laboratories for education in science, technology, and engineering: A review. Computers & Education,95 , 309–27. https://doi.org/https://doi.org/10.1016/j.compedu.2016.02.002
Rapp, D. N. (2005). Mental models: Theoretical issues for visualizations in science education. InVisualization in Science Education. Models and Modeling in Science Education , vol 1 . Ed. Gilbert, J. K., Springer, Dordrecht, Netherlands. https://doi.org/10.1007/1-4020-3613-2_4
Schaefli, B., Harman, C. J., Sivapalan, M., & Schymanski, S. J. (2011). HESS Opinions: Hydrologic predictions in a changing environment: behavioral modeling. Hydrology and Earth System Science, 15 , 635–646. https://doi.org/10.5194/hess-15-635-2011
Shen, C. (2018). A transdisciplinary review of deep learning research and its relevance for water resources scientists. Water Resources Research, 54 (11): 8558–8593. https://doi.org/10.1029/2018WR022643
Smith, J. D., & Lohani, V. K. (2019). Development of a Virtual Environment for Environmental Monitoring Education. In Smart Industry & Smart Education , Eds. Auer, M. E., & Langmann, R., vol. 47, 443–450, Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-95678-7_49
Stall, S., et al. (2018), Advancing FAIR data in Earth, space, and environmental science, Eos, 99,  https://doi.org/10.1029/2018EO109301. Published on 05 November 2018.
Šumak, B., Marjan H., & Maja, P. (2011). A meta-analysis of e-learning technology acceptance: The role of user types and e-learning technology types. Computers in Human Behavior , 27 (6), 2067–2077. https://doi.org/doi: 10.1016/j.chb.2011.08.005
Takatsuka, M., & Gahegan, M. (2002). GeoVISTA Studio: A codeless visual programming environment for geoscientific data analysis and visualization. Computers & Geosciences,  28 (10), 1131-1144. https://doi.org/10.1016/S0098-3004(02)00031-6
Figure 1: An example of a hierarchical structure to visualize sub-components of ecohydrologic models