Global scale hydrological modelling at 100 m, 1 h resolution, in Python

Author(s):  
Kor de Jong ◽  
Marc van Kreveld ◽  
Debabrata Panja ◽  
Oliver Schmitz ◽  
Derek Karssenberg

<p>Data availability at global scale is increasing exponentially. Although considerable challenges remain regarding the identification of model structure and parameters of continental scale hydrological models, we will soon reach the situation that global scale models could be defined at very high resolutions close to 100 m or less. One of the key challenges is how to make simulations of these ultra-high resolution models tractable ([1]).</p><p>Our research contributes by the development of a model building framework that is specifically designed to distribute calculations over multiple cluster nodes. This framework enables domain experts like hydrologists to develop their own large scale models, using a scripting language like Python, without the need to acquire the skills to develop low-level computer code for parallel and distributed computing.</p><p>We present the design and implementation of this software framework and illustrate its use with a prototype 100 m, 1 h continental scale hydrological model. Our modelling framework ensures that any model built with it is parallelized. This is made possible by providing the model builder with a set of building blocks of models, which are coded in such a manner that parallelization of calculations occurs within and across these building blocks, for any combination of building blocks. There is thus full flexibility on the side of the modeller, without losing performance.</p><p>This breakthrough is made possible by applying a novel approach to the implementation of the model building framework, called asynchronous many-tasks, provided by the HPX C++ software library ([3]). The code in the model building framework expresses spatial operations as large collections of interdependent tasks that can be executed efficiently on individual laptops as well as computer clusters ([2]). Our framework currently includes the most essential operations for building large scale hydrological models, including those for simulating transport of material through a flow direction network. By combining these operations, we rebuilt an existing 100 m, 1 h resolution model, thus far used for simulations of small catchments, requiring limited coding as we only had to replace the computational back end of the existing model. Runs at continental scale on a computer cluster show acceptable strong and weak scaling providing a strong indication that global simulations at this resolution will soon be possible, technically speaking.</p><p>Future work will focus on extending the set of modelling operations and adding scalable I/O, after which existing models that are currently limited in their ability to use the computational resources available to them can be ported to this new environment.</p><p>More information about our modelling framework is at https://lue.computationalgeography.org.</p><p><strong>References</strong></p><p>[1] M. Bierkens. Global hydrology 2015: State, trends, and directions. Water Resources Research, 51(7):4923–4947, 2015.<br>[2] K. de Jong, et al. An environmental modelling framework based on asynchronous many-tasks: scalability and usability. Submitted.<br>[3] H. Kaiser, et al. HPX - The C++ standard library for parallelism and concurrency. Journal of Open Source Software, 5(53):2352, 2020.</p>

2020 ◽  
Author(s):  
Tom Gleeson ◽  
Thorsten Wagener ◽  
Petra Döll ◽  
Samuel C. Zipper ◽  
Charles West ◽  
...  

Abstract. Continental- to global-scale hydrologic and land surface models increasingly include representations of the groundwater system, driven by crucial Earth science and sustainability problems. These models are essential for examining, communicating, and understanding the dynamic interactions between the Earth System above and below the land surface as well as the opportunities and limits of groundwater resources. A key question for this nascent and rapidly developing field is how to evaluate the realism and performance of such large-scale groundwater models given limitations in data availability and commensurability. Our objective is to provide clear recommendations for improving the evaluation of groundwater representation in continental- to global-scale models. We identify three evaluation approaches, including comparing model outputs with available observations of groundwater levels or other state or flux variables (observation-based evaluation); comparing several models with each other with or without reference to actual observations (model-based evaluation); and comparing model behavior with expert expectations of hydrologic behaviors that we expect to see in particular regions or at particular times (expert-based evaluation). Based on current and evolving practices in model evaluation as well as innovations in observations, machine learning and expert elicitation, we argue that combining observation-, model-, and expert-based model evaluation approaches may significantly improve the realism of groundwater representation in large-scale models, and thus our quantification, understanding, and prediction of crucial Earth science and sustainability problems. We encourage greater community-level communication and cooperation on these challenges, including among global hydrology and land surface modelers, local to regional hydrogeologists, and hydrologists focused on model development and evaluation.


2021 ◽  
Vol 14 (12) ◽  
pp. 7545-7571
Author(s):  
Tom Gleeson ◽  
Thorsten Wagener ◽  
Petra Döll ◽  
Samuel C. Zipper ◽  
Charles West ◽  
...  

Abstract. Continental- to global-scale hydrologic and land surface models increasingly include representations of the groundwater system. Such large-scale models are essential for examining, communicating, and understanding the dynamic interactions between the Earth system above and below the land surface as well as the opportunities and limits of groundwater resources. We argue that both large-scale and regional-scale groundwater models have utility, strengths, and limitations, so continued modeling at both scales is essential and mutually beneficial. A crucial quest is how to evaluate the realism, capabilities, and performance of large-scale groundwater models given their modeling purpose of addressing large-scale science or sustainability questions as well as limitations in data availability and commensurability. Evaluation should identify if, when, or where large-scale models achieve their purpose or where opportunities for improvements exist so that such models better achieve their purpose. We suggest that reproducing the spatiotemporal details of regional-scale models and matching local data are not relevant goals. Instead, it is important to decide on reasonable model expectations regarding when a large-scale model is performing “well enough” in the context of its specific purpose. The decision of reasonable expectations is necessarily subjective even if the evaluation criteria are quantitative. Our objective is to provide recommendations for improving the evaluation of groundwater representation in continental- to global-scale models. We describe current modeling strategies and evaluation practices, and we subsequently discuss the value of three evaluation strategies: (1) comparing model outputs with available observations of groundwater levels or other state or flux variables (observation-based evaluation), (2) comparing several models with each other with or without reference to actual observations (model-based evaluation), and (3) comparing model behavior with expert expectations of hydrologic behaviors in particular regions or at particular times (expert-based evaluation). Based on evolving practices in model evaluation as well as innovations in observations, machine learning, and expert elicitation, we argue that combining observation-, model-, and expert-based model evaluation approaches, while accounting for commensurability issues, may significantly improve the realism of groundwater representation in large-scale models, thus advancing our ability for quantification, understanding, and prediction of crucial Earth science and sustainability problems. We encourage greater community-level communication and cooperation on this quest, including among global hydrology and land surface modelers, local to regional hydrogeologists, and hydrologists focused on model development and evaluation.


2021 ◽  
Author(s):  
Tom Gleeson ◽  
Thorsten Wagener ◽  
Petra Döll ◽  
Samuel C. Zipper ◽  
Charles West ◽  
...  

Abstract. Continental- to global-scale hydrologic and land surface models increasingly include representations of the groundwater system. Such large-scale models are essential for examining, communicating, and understanding the dynamic interactions between the Earth System above and below the land surface as well as the opportunities and limits of groundwater resources. We argue that both large-scale and regional-scale groundwater models have utility, strengths and limitations so continued modeling at both scales is essential and mutually beneficial. A crucial quest is how to evaluate the realism, capabilities and performance of large-scale groundwater models given their modeling purpose of addressing large-scale science or sustainability questions as well as limitations in data availability and commensurability. Evaluation should identify if, when or where large-scale models achieve their purpose or where opportunities for improvements exists so that such models better achieve their purpose. We suggest that reproducing the spatio-temporal details of regional-scale models and matching local data is not a relevant goal. Instead, it is important to decide on reasonable model expectations regarding when a large scale model is performing “well enough” in the context of its specific purpose. The decision of reasonable expectations is necessarily subjective even if the evaluation criteria is quantitative. Our objective is to provide recommendations for improving the evaluation of groundwater representation in continental- to global-scale models. We describe current modeling strategies and evaluation practices, and subsequently discuss the value of three evaluation strategies: 1) comparing model outputs with available observations of groundwater levels or other state or flux variables (observation-based evaluation); 2) comparing several models with each other with or without reference to actual observations (model-based evaluation); and 3) comparing model behavior with expert expectations of hydrologic behaviors in particular regions or at particular times (expert-based evaluation). Based on evolving practices in model evaluation as well as innovations in observations, machine learning and expert elicitation, we argue that combining observation-, model-, and expert-based model evaluation approaches, while accounting for commensurability issues, may significantly improve the realism of groundwater representation in large-scale models. Thus advancing our ability for quantification, understanding, and prediction of crucial Earth science and sustainability problems. We encourage greater community-level communication and cooperation on this quest, including among global hydrology and land surface modelers, local to regional hydrogeologists, and hydrologists focused on model development and evaluation.


2016 ◽  
Author(s):  
Rogier Westerhoff ◽  
Paul White ◽  
Zara Rawlinson

Abstract. Large-scale models and satellite data are increasingly used to characterise groundwater and its recharge at the global scale. Although these models have the potential to fill in data gaps and solve trans-boundary issues, they are often neglected in smaller-scale studies, since data are often coarse or uncertain. Large-scale models and satellite data could play a more important role in smaller-scale (i.e., national or regional) studies, if they could be adjusted to fit that scale. In New Zealand, large-scale models and satellite data are not used for groundwater recharge estimation at the national scale, since regional councils (i.e., the water managers) have varying water policy and models are calibrated at the local scale. Also, some regions have many localised ground observations (but poor record coverage), whereas others are data-sparse. Therefore, estimation of recharge is inconsistent at the national scale. This paper presents an approach to apply large-scale, global, models and satellite data to estimate rainfall recharge at the national to regional scale across New Zealand. We present a model, NGRM, that is largely inspired by the global-scale WaterGAP recharge model, but is improved and adjusted using national data. The NGRM model uses MODIS-derived ET and vegetation satellite data, and the available nation-wide datasets on rainfall, elevation, soil and geology. A valuable addition to the recharge estimation is the model uncertainty estimate, based on variance, covariance and sensitivity of all input data components in the model environment. This research shows that, with minor model adjustments and use of improved input data, large-scale models and satellite data can be used to derive rainfall recharge estimates, including their uncertainty, at the smaller scale, i.e., national and regional scale of New Zealand. The estimated New Zealand recharge of the NGRM model compare well to most local and regional lysimeter data and recharge models. The NGRM is therefore assumed to be capable to fill in gaps in data-sparse areas and to create more consistency between datasets from different regions, i.e., to solve trans-boundary issues. This research also shows that smaller-scale recharge studies in New Zealand should include larger boundaries than only a (sub-)aquifer, and preferably the whole catchment. This research points out the need for improved collaboration on the international to national to regional levels to further merge large-scale (global) models to smaller (i.e., national or regional) scales. Future research topics should, collaboratively, focus on: improvement of rainfall-runoff and snowmelt methods; inclusion of river recharge; further improvement of input data (rainfall, evapotranspiration, soil and geology); and the impact of recharge uncertainty in mountainous and irrigated areas.


2012 ◽  
Vol 9 (7) ◽  
pp. 8375-8424 ◽  
Author(s):  
A. F. Van Loon ◽  
M. H. J. Van Huijgevoort ◽  
H. A. J. Van Lanen

Abstract. Hydrological drought is increasingly studied using large-scale models. It is, however, not sure whether large-scale models reproduce the development of hydrological drought correctly. The pressing question is: how well do large-scale models simulate the propagation from meteorological to hydrological drought? To answer this question, we evaluated the simulation of drought propagation in an ensemble mean of ten large-scale models, both land-surface models and global hydrological models, that were part of the model intercomparison project of WATCH (WaterMIP). For a selection of case study areas, we studied drought characteristics (number of droughts, duration, severity), drought propagation features (pooling, attenuation, lag, lengthening), and hydrological drought typology (classical rainfall deficit drought, rain-to-snow-season drought, wet-to-dry-season drought, cold snow season drought, warm snow season drought, composite drought). Drought characteristics simulated by large-scale models clearly reflected drought propagation, i.e. drought events became less and longer when moving through the hydrological cycle. However, more differentiation was expected between fast and slowly responding systems, with slowly responding systems having less and longer droughts in runoff than fast responding systems. This was not found using large-scale models. Drought propagation features were poorly reproduced by the large-scale models, because runoff reacted immediately to precipitation, in all case study areas. This fast reaction to precipitation, even in cold climates in winter and in semi-arid climates in summer, also greatly influenced the hydrological drought typology as identified by the large-scale models. In general, the large-scale models had the correct representation of drought types, but the percentages of occurrence had some important mismatches, e.g. an overestimation of classical rainfall deficit droughts, and an underestimation of wet-to-dry-season droughts and snow-related droughts. Furthermore, almost no composite droughts were simulated for slowly responding areas, while many multi-year drought events were expected in these systems. We conclude that drought propagation processes are reasonably well reproduced by the ensemble mean of large-scale models in contrasting catchments in Europe and that some challenges remain in catchments with cold and semi-arid climates and catchments with large storage in aquifers or lakes. Improvement of drought simulation in large-scale models should focus on a better representation of hydrological processes that are important for drought development, such as evapotranspiration, snow accumulation and melt, and especially storage. Besides the more explicit inclusion of storage (e.g. aquifers) in large-scale models, also parametrisation of storage processes requires attention, for example through a global scale dataset on aquifer characteristics.


2021 ◽  
Author(s):  
Alena Bartosova ◽  
Berit Arheimer ◽  
Alban de Lavenne ◽  
René Capell ◽  
Johan Strömqvist

<p>Continental and global dynamic hydrological models have emerged recently as tools for e.g. flood forecasting, large-scale climate impact analyses, and estimation of time-dynamic water fluxes into sea basins. One such tool is a dynamic process-based rainfall-runoff and water quality model Hydrological Predictions for Environment (HYPE). We present and compare historical simulations of runoff, soil moisture, aridity, and sediment concentrations for three nested model domains using global, continental (Europe), and national (Sweden) catchment-based HYPE applications. Future impacts on hydrological variables from changing climate were then assessed using the global and continental HYPE applications with ensembles based on 3 CMIP5 global climate models (GCMs).</p><p>Simulated historical sediment concentrations varied considerably among the nested models in spatial patterns while runoff values were more similar. Regardless of the variation, the global model was able to provide information on climate change impacts comparable to those from the continental and national models for hydrological indicators. Output variables that were calibrated, e.g. runoff, were shown to result in more reliable and consistent projected changes among the different model scales than derived variables such as the actual aridity index. The comparison was carried out for ensemble averages as well as individual GCMs to illustrate the variability and the need for robust assessments.</p><p>Global hydrological models are shown to be valuable tools for e.g. first screenings of climate change effects and detection of spatial patterns and can be useful to provide information on current and future hydrological states at various domains. The challenge is (1) in deciding when we should use the large-scale models and (2) in interpreting the results, considering the uncertainty of the model results and quality of data especially at the global scale. Comparison across nested domains demonstrates the significance of scale which needs to be considered when interpreting the impacts alongside with model performance.</p><p>Bartosova et al, 2021: Large-scale hydrological and sediment modeling in nested domains under current and changing climate. Accepted to Special Issue Journal of Hydraulic Engineering.</p>


2012 ◽  
Vol 16 (11) ◽  
pp. 4057-4078 ◽  
Author(s):  
A. F. Van Loon ◽  
M. H. J. Van Huijgevoort ◽  
H. A. J. Van Lanen

Abstract. Hydrological drought is increasingly studied using large-scale models. It is, however, not sure whether large-scale models reproduce the development of hydrological drought correctly. The pressing question is how well do large-scale models simulate the propagation from meteorological to hydrological drought? To answer this question, we evaluated the simulation of drought propagation in an ensemble mean of ten large-scale models, both land-surface models and global hydrological models, that participated in the model intercomparison project of WATCH (WaterMIP). For a selection of case study areas, we studied drought characteristics (number of droughts, duration, severity), drought propagation features (pooling, attenuation, lag, lengthening), and hydrological drought typology (classical rainfall deficit drought, rain-to-snow-season drought, wet-to-dry-season drought, cold snow season drought, warm snow season drought, composite drought). Drought characteristics simulated by large-scale models clearly reflected drought propagation; i.e. drought events became fewer and longer when moving through the hydrological cycle. However, more differentiation was expected between fast and slowly responding systems, with slowly responding systems having fewer and longer droughts in runoff than fast responding systems. This was not found using large-scale models. Drought propagation features were poorly reproduced by the large-scale models, because runoff reacted immediately to precipitation, in all case study areas. This fast reaction to precipitation, even in cold climates in winter and in semi-arid climates in summer, also greatly influenced the hydrological drought typology as identified by the large-scale models. In general, the large-scale models had the correct representation of drought types, but the percentages of occurrence had some important mismatches, e.g. an overestimation of classical rainfall deficit droughts, and an underestimation of wet-to-dry-season droughts and snow-related droughts. Furthermore, almost no composite droughts were simulated for slowly responding areas, while many multi-year drought events were expected in these systems. We conclude that most drought propagation processes are reasonably well reproduced by the ensemble mean of large-scale models in contrasting catchments in Europe. Challenges, however, remain in catchments with cold and semi-arid climates and catchments with large storage in aquifers or lakes. This leads to a high uncertainty in hydrological drought simulation at large scales. Improvement of drought simulation in large-scale models should focus on a better representation of hydrological processes that are important for drought development, such as evapotranspiration, snow accumulation and melt, and especially storage. Besides the more explicit inclusion of storage in large-scale models, also parametrisation of storage processes requires attention, for example through a global-scale dataset on aquifer characteristics, improved large-scale datasets on other land characteristics (e.g. soils, land cover), and calibration/evaluation of the models against observations of storage (e.g. in snow, groundwater).


2018 ◽  
Author(s):  
Abbie Chapman ◽  
Amanda E Bates ◽  
Verena Tunnicliffe ◽  
The sFDvent Working Group ◽  

The taxonomic composition of hydrothermal vent communities differs markedly on a global scale, forming distinct biogeographic provinces. The relative biodiversity of these areas can be assessed using traits as a common, cross-province ‘currency’. First, we used well-studied Juan de Fuca Ridge vents (NE Pacific) to assess trait data availability for vent species and to test the performance of functional diversity metrics given a species-poor system. These investigations highlighted vents as model ‘untouched’ ecosystems for developing ecological theory for conservation, advocating the potential of a vent trait database. Next, we built a global trait database for vent species – ‘sFDvent’. We selected traits that characterized the performance of a species and its contribution to ecosystem function, and best matched with established trait databases to ensure cross-ecosystem consistency. An international pool of experts scored these traits to populate the 14-trait database. Using sFDvent, we: created the first map of functional biogeography for deep-sea hydrothermal vents; assessed global-scale functional biodiversity trends (e.g., the East Pacific has fewer functionally unique species than the West Pacific, based on preliminary analyses); and evaluated the potential roles of large-scale environmental processes on these patterns.


2017 ◽  
Author(s):  
Chinchu Mohan ◽  
Andrew W. Western ◽  
Yongping Wei ◽  
Margarita Saft

Abstract. Groundwater recharge is one of the important factors determining the groundwater development potential of an area. Even though recharge plays a key role in controlling groundwater system dynamics, much uncertainty remains regarding the relationships between groundwater recharge and its governing factors at a large scale. The aims of this study were to identify the most influential factors on groundwater recharge, and to develop an empirical model to estimate diffuse rainfall recharge at a global-scale. Recharge estimates reported in the literature from various parts of the world (715 sites) were compiled and used in model building and testing exercises. Unlike conventional recharge estimates from water balance, this study used a multimodel inference approach and information theory to explain the relation between groundwater recharge and influential factors, and to predict groundwater recharge at 0.50 resolution. The results show that meteorological factors (precipitation and potential evapotranspiration) and vegetation factors (land use and land cover) had the most predictive power for recharge. According to the model, long term global average annual recharge (1981–2014) was 134 mm/yr with a prediction error ranging from −8 mm/yr to 10 mm/yr for 97.2 % of cases. The recharge estimates presented in this study are unique and more reliable than the existing global groundwater recharge estimates because of the extensive validation carried out using both independent local estimates collated from the literature and national statistics from Food and Agriculture Organisation (FAO). In a water scarce future driven by increased anthropogenic development, the results from this study will aid in making informed decision about groundwater potential at a large scale.


2020 ◽  
Vol 163 (3) ◽  
pp. 1121-1141
Author(s):  
Valentina Krysanova ◽  
Fred F. Hattermann ◽  
Zbigniew W. Kundzewicz

AbstractThis paper introduces the Special Issue (SI) “How evaluation of hydrological models influences results of climate impact assessment.” The main objectives were as follows: (a) to test a comprehensive model calibration/validation procedure, consisting of five steps, for regional-scale hydrological models; (b) to evaluate performance of global-scale hydrological models; and (c) to reveal whether the calibration/validation methods and the model evaluation results influence climate impacts in terms of the magnitude of the change signal and the uncertainty range. Here, we shortly describe the river basins and large regions used as case studies; the hydrological models, data, and climate scenarios used in the studies; and the applied approaches for model evaluation and for analysis of projections for the future. After that, we summarize the main findings. The following general conclusions could be drawn. After successful comprehensive calibration and validation, the regional-scale models are more robust and their projections for the future differ from those of the model versions after the conventional calibration and validation. Therefore, climate impacts based on the former models are more trustworthy than those simulated by the latter models. Regarding the global-scale models, using only models with satisfactory or good performance on historical data and weighting them based on model evaluation results is a more reliable approach for impact assessment compared to the ensemble mean approach that is commonly used. The former method provides impact results with higher credibility and reduced spreads in comparison to the latter approach. The studies for this SI were performed in the framework of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP).


Sign in / Sign up

Export Citation Format

Share Document