Transfer learning applications in hydrologic modeling

2020 ◽  
Author(s):  
Joseph Hamman ◽  
Andrew Bennett
Keyword(s):  
Transfer Learning ◽  
Short Term Memory ◽  
Hydrologic Model ◽  
Surface Fluxes ◽  
Watershed Scale ◽  
Streamflow Prediction ◽  
Data Set ◽  
Hydrologic Prediction ◽  
New Research ◽  
Combine Information

<p>Early work in the field of Machine Learning (ML) for hydrologic prediction is showing significant potential. Indeed, it has provided important and measurable advances toward prediction in ungauged basins (PUB). At the same time, it has motivated a new research targeting important ML topics such as uncertainty attribution and physical constrains. It has also brought into question how to best harness the wide variety of climatic and hydrologic data available today. In this work, we present initial results employing transfer learning to combine information about meteorology, streamflow, surface fluxes (FluxNet), and snow (SNOTEL) into a state of the art ML-based hydrologic model. Specifically, we will present early work demonstrating how relatively simple implementations of transfer learning can be used to enhance predictions of streamflow by transferring learning from flux and snow station observations to the watershed scale. Our work is shown to extend recently published results from Kratzert et al. (2018) using the CAMELS data set (Newman et al. 2014) for streamflow prediction in North America.</p><ul><li>Kratzert, F., Klotz, D., Brenner, C., Schulz, K., and Herrnegger, M.: Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks, Hydrol. Earth Syst. Sci., 22, 6005-6022, https://doi.org/10.5194/hess-22-6005-2018, 2018a.</li> <li>Newman; K. Sampson; M. P. Clark; A. Bock; R. J. Viger; D. Blodgett, 2014. A large-sample watershed-scale hydrometeorological dataset for the contiguous USA. Boulder, CO: UCAR/NCAR. https://dx.doi.org/10.5065/D6MW2F4D</li> </ul>

scholarly journals A Dynamical Climate Model–Driven Hydrologic Prediction System for the Fraser River, Canada

2015 ◽  
Vol 16 (3) ◽  
pp. 1273-1292 ◽  
Author(s):  
Rajesh R. Shrestha ◽  
Markus A. Schnorbus ◽  
Alex J. Cannon
Keyword(s):  
Climate Models ◽  
Climate Model ◽  
Hydrologic Model ◽  
Prediction Skill ◽  
Fraser River ◽  
Prediction System ◽  
Streamflow Prediction ◽  
Model Driven ◽  
Hydrologic Prediction ◽  
Seasonal Streamflow

Abstract Recent improvements in forecast skill of the climate system by dynamical climate models could lead to improvements in seasonal streamflow predictions. This study evaluates the hydrologic prediction skill of a dynamical climate model–driven hydrologic prediction system (CM-HPS), based on an ensemble of statistically downscaled outputs from the Canadian Seasonal to Interannual Prediction System (CanSIPS). For comparison, historical and future climate traces–driven ensemble streamflow prediction (ESP) was employed. The Variable Infiltration Capacity model (VIC) hydrologic model setup for the Fraser River basin, British Columbia, Canada, was used as a test bed for the two systems. In both cases, results revealed limited precipitation prediction skill. For streamflow prediction, the ESP approach has very limited or no correlation skill beyond the months influenced by initial hydrologic conditions, while the CM-HPS has moderately better correlation skill, attributable to the enhanced temperature prediction skill that results from CanSIPS’s ability to predict El Niño–Southern Oscillation (ENSO) and its teleconnections. The root-mean-square error, bias, and categorical skills for the two methods are mostly similar. Hydrologic modeling uncertainty also affects the prediction skill, and in some cases prediction skill is constrained by hydrologic model skill. Overall, the CM-HPS shows potential for seasonal streamflow prediction, and further enhancements in climate models could potentially to lead to more skillful hydrologic predictions.

A Deep Learning based Arabic Script Recognition System: Benchmark on KHAT

2020 ◽  
Vol 17 (3) ◽  
pp. 299-305 ◽  
Author(s):  
Riaz Ahmad ◽  
Saeeda Naz ◽  
Muhammad Afzal ◽  
Sheikh Rashid ◽  
Marcus Liwicki ◽  
...  
Keyword(s):  
Deep Learning ◽  
Character Recognition ◽  
Data Augmentation ◽  
Short Term Memory ◽  
Recognition System ◽  
Learning Approach ◽  
Arabic Text ◽  
Data Set ◽  
Processing Step ◽  
Handwritten Arabic

This paper presents a deep learning benchmark on a complex dataset known as KFUPM Handwritten Arabic TexT (KHATT). The KHATT data-set consists of complex patterns of handwritten Arabic text-lines. This paper contributes mainly in three aspects i.e., (1) pre-processing, (2) deep learning based approach, and (3) data-augmentation. The pre-processing step includes pruning of white extra spaces plus de-skewing the skewed text-lines. We deploy a deep learning approach based on Multi-Dimensional Long Short-Term Memory (MDLSTM) networks and Connectionist Temporal Classification (CTC). The MDLSTM has the advantage of scanning the Arabic text-lines in all directions (horizontal and vertical) to cover dots, diacritics, strokes and fine inflammation. The data-augmentation with a deep learning approach proves to achieve better and promising improvement in results by gaining 80.02% Character Recognition (CR) over 75.08% as baseline.

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

2019 ◽  
Vol 9 (1) ◽  
pp. 16-31
Author(s):  
Kyungkoo Jun
Keyword(s):  
Fourier Transform ◽  
Deep Learning ◽  
Short Term Memory ◽  
Window Size ◽  
Sensor Data ◽  
Data Sets ◽  
Data Set ◽  
Proposed Model ◽  
Testing Data ◽  
Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

scholarly journals Leveraging Label Information in a Knowledge-Driven Approach for Rolling-Element Bearings Remaining Useful Life Prediction

Energies ◽  
2021 ◽  
Vol 14 (8) ◽  
pp. 2163
Author(s):  
Tarek Berghout ◽  
Mohamed Benbouzid ◽  
Leïla-Hayet Mouss
Keyword(s):  
Transfer Learning ◽  
Short Term Memory ◽  
Remaining Useful Life ◽  
Accelerated Life Tests ◽  
Learning Path ◽  
Accelerated Life ◽  
Unseen Data ◽  
Label Information ◽  
Life Tests ◽  
Ill Posed

Since bearing deterioration patterns are difficult to collect from real, long lifetime scenarios, data-driven research has been directed towards recovering them by imposing accelerated life tests. Consequently, insufficiently recovered features due to rapid damage propagation seem more likely to lead to poorly generalized learning machines. Knowledge-driven learning comes as a solution by providing prior assumptions from transfer learning. Likewise, the absence of true labels was able to create inconsistency related problems between samples, and teacher-given label behaviors led to more ill-posed predictors. Therefore, in an attempt to overcome the incomplete, unlabeled data drawbacks, a new autoencoder has been designed as an additional source that could correlate inputs and labels by exploiting label information in a completely unsupervised learning scheme. Additionally, its stacked denoising version seems to more robustly be able to recover them for new unseen data. Due to the non-stationary and sequentially driven nature of samples, recovered representations have been fed into a transfer learning, convolutional, long–short-term memory neural network for further meaningful learning representations. The assessment procedures were benchmarked against recent methods under different training datasets. The obtained results led to more efficiency confirming the strength of the new learning path.

scholarly journals Classification of jujube defects in small data sets based on transfer learning

2021 ◽  
Author(s):  
Jianping Ju ◽  
Hong Zheng ◽  
Xiaohang Xu ◽  
Zhongyuan Guo ◽  
Zhaohui Zheng ◽  
...  
Keyword(s):  
Transfer Learning ◽  
Loss Function ◽  
Training Model ◽  
Parameter Distribution ◽  
Test Accuracy ◽  
Small Data ◽  
Data Sets ◽  
Data Set ◽  
Small Data Sets

AbstractAlthough convolutional neural networks have achieved success in the field of image classification, there are still challenges in the field of agricultural product quality sorting such as machine vision-based jujube defects detection. The performance of jujube defect detection mainly depends on the feature extraction and the classifier used. Due to the diversity of the jujube materials and the variability of the testing environment, the traditional method of manually extracting the features often fails to meet the requirements of practical application. In this paper, a jujube sorting model in small data sets based on convolutional neural network and transfer learning is proposed to meet the actual demand of jujube defects detection. Firstly, the original images collected from the actual jujube sorting production line were pre-processed, and the data were augmented to establish a data set of five categories of jujube defects. The original CNN model is then improved by embedding the SE module and using the triplet loss function and the center loss function to replace the softmax loss function. Finally, the depth pre-training model on the ImageNet image data set was used to conduct training on the jujube defects data set, so that the parameters of the pre-training model could fit the parameter distribution of the jujube defects image, and the parameter distribution was transferred to the jujube defects data set to complete the transfer of the model and realize the detection and classification of the jujube defects. The classification results are visualized by heatmap through the analysis of classification accuracy and confusion matrix compared with the comparison models. The experimental results show that the SE-ResNet50-CL model optimizes the fine-grained classification problem of jujube defect recognition, and the test accuracy reaches 94.15%. The model has good stability and high recognition accuracy in complex environments.

Event detection of different English data sources based on transfer learning

2021 ◽  
pp. 1-11
Author(s):  
Yanan Huang ◽  
Yuji Miao ◽  
Zhenjing Da
Keyword(s):  
Transfer Learning ◽  
Event Detection ◽  
Visual Analysis ◽  
Learning Algorithm ◽  
Data Sources ◽  
Data Set ◽  
Data Source ◽  
Single Data Source ◽  
The Difference ◽  
Single Data

The methods of multi-modal English event detection under a single data source and isomorphic event detection of different English data sources based on transfer learning still need to be improved. In order to improve the efficiency of English and data source time detection, based on the transfer learning algorithm, this paper proposes multi-modal event detection under a single data source and isomorphic event detection based on transfer learning for different data sources. Moreover, by stacking multiple classification models, this paper makes each feature merge with each other, and conducts confrontation training through the difference between the two classifiers to further make the distribution of different source data similar. In addition, in order to verify the algorithm proposed in this paper, a multi-source English event detection data set is collected through a data collection method. Finally, this paper uses the data set to verify the method proposed in this paper and compare it with the current most mainstream transfer learning methods. Through experimental analysis, convergence analysis, visual analysis and parameter evaluation, the effectiveness of the algorithm proposed in this paper is demonstrated.

Auto-sharing parameters for transfer learning based on multi-objective optimization

2021 ◽  
pp. 1-13
Author(s):  
Hailin Liu ◽  
Fangqing Gu ◽  
Zixian Lin
Keyword(s):  
Transfer Learning ◽  
Optimization Problem ◽  
Data Sets ◽  
Multi Objective Optimization ◽  
Particle Swarm Optimizer ◽  
Real World Data ◽  
Data Set ◽  
Target Task ◽  
Main Research ◽  
Multi Objective

Transfer learning methods exploit similarities between different datasets to improve the performance of the target task by transferring knowledge from source tasks to the target task. “What to transfer” is a main research issue in transfer learning. The existing transfer learning method generally needs to acquire the shared parameters by integrating human knowledge. However, in many real applications, an understanding of which parameters can be shared is unknown beforehand. Transfer learning model is essentially a special multi-objective optimization problem. Consequently, this paper proposes a novel auto-sharing parameter technique for transfer learning based on multi-objective optimization and solves the optimization problem by using a multi-swarm particle swarm optimizer. Each task objective is simultaneously optimized by a sub-swarm. The current best particle from the sub-swarm of the target task is used to guide the search of particles of the source tasks and vice versa. The target task and source task are jointly solved by sharing the information of the best particle, which works as an inductive bias. Experiments are carried out to evaluate the proposed algorithm on several synthetic data sets and two real-world data sets of a school data set and a landmine data set, which show that the proposed algorithm is effective.

scholarly journals Forecasting Energy Consumption of Wastewater Treatment Plants with a Transfer Learning Approach for Sustainable Cities

Electronics ◽  
2021 ◽  
Vol 10 (10) ◽  
pp. 1149
Author(s):  
Pedro Oliveira ◽  
Bruno Fernandes ◽  
Cesar Analide ◽  
Paulo Novais
Keyword(s):  
Wastewater Treatment ◽  
Energy Consumption ◽  
Transfer Learning ◽  
Short Term Memory ◽  
Wastewater Treatment Plants ◽  
Learning Approach ◽  
Sustainable Cities ◽  
Short Term ◽  
Long Short Term Memory ◽  
Gated Recurrent Units

A major challenge of today’s society is to make large urban centres more sustainable. Improving the energy efficiency of the various infrastructures that make up cities is one aspect being considered when improving their sustainability, with Wastewater Treatment Plants (WWTPs) being one of them. Consequently, this study aims to conceive, tune, and evaluate a set of candidate deep learning models with the goal being to forecast the energy consumption of a WWTP, following a recursive multi-step approach. Three distinct types of models were experimented, in particular, Long Short-Term Memory networks (LSTMs), Gated Recurrent Units (GRUs), and uni-dimensional Convolutional Neural Networks (CNNs). Uni- and multi-variate settings were evaluated, as well as different methods for handling outliers. Promising forecasting results were obtained by CNN-based models, being this difference statistically significant when compared to LSTMs and GRUs, with the best model presenting an approximate overall error of 630 kWh when on a multi-variate setting. Finally, to overcome the problem of data scarcity in WWTPs, transfer learning processes were implemented, with promising results being achieved when using a pre-trained uni-variate CNN model, with the overall error reducing to 325 kWh.

On the benefit of the Canadian Small Lake Model to better represent the impact of small natural lakes on GEM-Hydro streamflow simulations.

2021 ◽  
Author(s):  
Etienne Gaborit ◽  
Murray MacKay ◽  
Camille Garnaud ◽  
Vincent Fortin
Keyword(s):  
Water Balance ◽  
Grid Point ◽  
Hydrologic Model ◽  
Surface Fluxes ◽  
Dew Point ◽  
Multiple Model ◽  
Small Lake ◽  
Surface Component ◽  
Lake Model ◽  
The Impact

<p>This study aims at assessing the impact of a new lake model on streamflow simulations performed with the GEM-Hydro hydrologic model developed at ECCC. GEM-Hydro is at the heart of the National Surface and River Prediction System (NSRPS) which ECCC uses to forecast river flows over most of Canada. The GEM-Hydro model mainly consists of the GEM-Surf component to represent surface processes, and of the Watroute model to represent river and lake routing, in order to perform streamflow simulations and forecasts. The surface component of GEM-Hydro can simulate 5 different types of surfaces.  Currently, the water tile consists of a very simple algorithm which, in terms of water balance, consists of producing runoff fluxes simply equal to precipitation minus evaporation. This runoff over water surfaces is then provided as input, along with runoff and drainage generated over other surface tiles, to the Watroute model. The Watroute version used in GEM-Hydro currently only represents major lakes (area greater than 100km<sup>2</sup>) along the river networks, and does not represent the impact that small lakes can have on streamflow, which mainly consists in slowing down runoff before it reaches the main streams of the network.</p><p>Recently, the Canadian Small Lake Model (CSLM) was implemented in the surface component of GEM-Hydro to represent the energy and water balance over water tiles more accurately. So far, CSLM simulations have been shown promising in terms of evaporation, ice cover, absolute and dew point temperature simulations, compared with the former algorithm used over water. However, the impact of CSLM on the resulting streamflow simulations performed with GEM-Hydro has not been evaluated yet. This study aims first at evaluating the impact of CSLM on streamflow simulations, and secondly at testing different CSLM configurations as well as different coupling strategies with Watroute, with the objective of finding the best set up for the prediction of streamflow in Canada. For example, overland runoff generated by the land tile can be provided to the water tile of the same grid point in different ways, and the outflow computed at the outlet of the water tile can be computed with different parameters. Moreover, different outflow computations have to be taken into account depending on if the water tile of a grid point represents subgrid-scale lakes, or if on the contrary it belongs to a lake spanning over multiple model grid points.</p><p>To do so, different GEM-Hydro open-loop simulations have been performed on the Lake of the Woods watershed, located in Canada, with and without CSLM to represent water tiles. The CSLM configurations leading to the best results are presented here. CSLM simulations are also evaluated in terms of surface fluxes, to ensure that the main purpose of the model, which is to improve surface fluxes to ultimately improve atmospheric forecasts, is preserved, compared to the default configuration of the model. Ideas for further improving the coupling between the GEM-Hydro surface and routing components, in terms of lake processes, are also presented and will be tested in future work.</p>

ECLand: an ECMWF land surface modelling platform

2021 ◽  
Author(s):  
Gianpaolo Balsamo ◽  
Souhail Boussetta
Keyword(s):  
Land Surface ◽  
Urban Areas ◽  
Land Surface Model ◽  
Open Water ◽  
Surface Fluxes ◽  
Surface Model ◽  
New Developments ◽  
Surface Modelling ◽  
Land Surface Modelling ◽  
New Research

<p>The ECMWF operational land surface model, based on the Carbon-Hydrology Tiled ECMWF Scheme for Surface Exchanges over Land (CHTESSEL) is the baseline for global weather, climate and environmental applications at ECMWF. In order to expedite its progress and benefit from international collaboration, an ECLand platform has been designed to host advanced and modular schemes. ECLand is paving the way toward a land model that could support a wider range of modelling applications, facilitating global kilometer scales testing as envisaged in the Copernicus and Destination Earth programmes. This presentation introduces the CHTESSEL and its recent new developments that aims at hosting new research applications.</p><p>These new improvements touch upon different components of the model: (i) vegetation, (ii) snow, (iii) soil hydrology, (iv) open water/lakes (v) rivers and (vi) urban areas. The developments are evaluated separately with either offline simulations or coupled experiments, depending on their level of operational readiness, illustrating the benchmarking criteria for assessing process fidelity with regards to land surface fluxes and reservoirs involved in water-energy-carbon exchange, and within the Earth system prediction framework, as foreseen to enter upcoming ECMWF operational cycles.</p><p>Reference: Souhail Boussetta, Gianpaolo Balsamo*, Anna Agustì-Panareda, Gabriele Arduini, Anton Beljaars, Emanuel Dutra, Glenn Carver, Margarita Choulga, Ioan Hadade, Cinzia Mazzetti, Joaquìn Munõz-Sabater, Joe McNorton, Christel Prudhomme, Patricia De Rosnay, Irina Sandu, Nils Wedi, Dai Yamazaki, Ervin Zsoter, 2021: ECLand: an ECMWF land surface modelling platform, MDPI Atmosphere, (in prep).</p>

Sign in / Sign up

Export Citation Format

Share Document