scholarly journals A model-based approach to adjust microwave observations for operational applications: results of a campaign at Munich Airport in winter 2011/2012

2013 ◽  
Vol 6 (10) ◽  
pp. 2879-2891 ◽  
Author(s):  
J. Güldner
Keyword(s):  
Neural Network ◽  
Weather Prediction ◽  
Microwave Radiometer ◽  
Regression Method ◽  
Training Data ◽  
Data Set ◽  
Numerical Weather ◽  
Model Based ◽  
Temperature And Humidity ◽  
Airport Site

Abstract. In the frame of the project "LuFo iPort VIS" which focuses on the implementation of a site-specific visibility forecast, a field campaign was organised to offer detailed information to a numerical fog model. As part of additional observing activities, a 22-channel microwave radiometer profiler (MWRP) was operating at the Munich Airport site in Germany from October 2011 to February 2012 in order to provide vertical temperature and humidity profiles as well as cloud liquid water information. Independently from the model-related aims of the campaign, the MWRP observations were used to study their capabilities to work in operational meteorological networks. Over the past decade a growing quantity of MWRP has been introduced and a user community (MWRnet) was established to encourage activities directed at the set up of an operational network. On that account, the comparability of observations from different network sites plays a fundamental role for any applications in climatology and numerical weather forecast. In practice, however, systematic temperature and humidity differences (bias) between MWRP retrievals and co-located radiosonde profiles were observed and reported by several authors. This bias can be caused by instrumental offsets and by the absorption model used in the retrieval algorithms as well as by applying a non-representative training data set. At the Lindenberg observatory, besides a neural network provided by the manufacturer, a measurement-based regression method was developed to reduce the bias. These regression operators are calculated on the basis of coincident radiosonde observations and MWRP brightness temperature (TB) measurements. However, MWRP applications in a network require comparable results at just any site, even if no radiosondes are available. The motivation of this work is directed to a verification of the suitability of the operational local forecast model COSMO-EU of the Deutscher Wetterdienst (DWD) for the calculation of model-based regression operators in order to provide unbiased vertical profiles during the campaign at Munich Airport. The results of this algorithm and the retrievals of a neural network, specially developed for the site, are compared with radiosondes from Oberschleißheim located about 10 km apart from the MWRP site. Outstanding deviations for the lowest levels between 50 and 100 m are discussed. Analogously to the airport experiment, a model-based regression operator was calculated for Lindenberg and compared with both radiosondes and operational results of observation-based methods. The bias of the retrievals could be considerably reduced and the accuracy, which has been assessed for the airport site, is quite similar to those of the operational radiometer site at Lindenberg above 1 km height. Additional investigations are made to determine the length of the training period necessary for generating best estimates. Thereby three months have proven to be adequate. The results of the study show that on the basis of numerical weather prediction (NWP) model data, available everywhere at any time, the model-based regression method is capable of providing comparable results at a multitude of sites. Furthermore, the approach offers auspicious conditions for automation and continuous updating.

scholarly journals A model-based approach to adjust microwave observations for operational applications: results of a campaign at Munich Airport in winter 2011/2012

2013 ◽  
Vol 6 (2) ◽  
pp. 2935-2954 ◽  
Author(s):  
J. Güldner
Keyword(s):  
Neural Network ◽  
Weather Prediction ◽  
Microwave Radiometer ◽  
Weather Forecast ◽  
Forecast Model ◽  
Regression Method ◽  
Numerical Weather ◽  
Model Based ◽  
Temperature And Humidity ◽  
Airport Site

Abstract. In the frame of the project "LuFo iPort VIS" which focuses on the implementation of a site specific visibility forecast, a field campaign was organised to offer detailed information to a numerical fog model. As part of additional observing activities a 22-channel microwave radiometer profiler (MWRP) was operating at the Munich Airport site in Germany from October 2011 to February 2012 in order to provide vertical temperature and humidity profiles as well as cloud liquid water information. Independently from the model-related aims of the campaign, the MWRP observations were used to study their capabilities to work in operational meteorological networks. Over the past decade a growing quantity of MWRP has been introduced and a user community (MWRnet) was established to encourage activities directed at the set up of an operational network. On that account, the comparability of observations from different network sites plays a fundamental role for any applications in climatology and numerical weather forecast. In practice, however, systematic temperature and humidity differences (bias) between MWRP retrievals and co-located radiosonde profiles were observed and reported by several authors. This bias can be caused by instrumental offsets as well as by the absorption model used in the retrieval algorithms. At the Lindenberg observatory besides a neural network provided by the manufacturer, a measurement-based regression method was developed to reduce the bias. These regression operators are calculated on the basis of coincident radiosonde observations and MWRP brightness temperature (TB) measurements. However, MWRP applications in a network require comparable results at just any site, even if no radiosondes are available. The motivation of this work is directed to a verification of the suitability of the operational local forecast model COSMO-EU of the Deutscher Wetterdienst (DWD) for the calculation of model-based regression operators in order to provide unbiased vertical profiles during the campaign at Munich Airport. The results of this algorithm and the retrievals of a neural network, specially developed for the site, are compared with radiosondes from Oberschleißheim located about 10 km apart from the MWRP site. The bias of the retrievals could be considerably reduced and the accuracy, which has been assessed for the airport site, is quite similar to those of the operational radiometer site at Lindenberg above 1 km height. Additional investigations are made to determine the length of the training period necessary for generating best estimates. Thereby three months have proven to be adequate. The results of the study show that on the basis of numerical weather prediction (NWP) model data, available everywhere at any time, the model-based regression method is capable to provide comparable results at a multitude of sites. Furthermore, the approach offers auspicious conditions for automation and continuous updating.

scholarly journals An assessment of Bayesian bias estimator for numerical weather prediction

2008 ◽  
Vol 15 (6) ◽  
pp. 1013-1022 ◽  
Author(s):  
J. Son ◽  
D. Hou ◽  
Z. Toth
Keyword(s):  
Time Series ◽  
Numerical Weather Prediction ◽  
Sampling Error ◽  
Weather Prediction ◽  
Training Data ◽  
Statistical Characteristics ◽  
Forecast Errors ◽  
Data Sets ◽  
Data Set ◽  
Numerical Weather

Abstract. Various statistical methods are used to process operational Numerical Weather Prediction (NWP) products with the aim of reducing forecast errors and they often require sufficiently large training data sets. Generating such a hindcast data set for this purpose can be costly and a well designed algorithm should be able to reduce the required size of these data sets. This issue is investigated with the relatively simple case of bias correction, by comparing a Bayesian algorithm of bias estimation with the conventionally used empirical method. As available forecast data sets are not large enough for a comprehensive test, synthetically generated time series representing the analysis (truth) and forecast are used to increase the sample size. Since these synthetic time series retained the statistical characteristics of the observations and operational NWP model output, the results of this study can be extended to real observation and forecasts and this is confirmed by a preliminary test with real data. By using the climatological mean and standard deviation of the meteorological variable in consideration and the statistical relationship between the forecast and the analysis, the Bayesian bias estimator outperforms the empirical approach in terms of the accuracy of the estimated bias, and it can reduce the required size of the training sample by a factor of 3. This advantage of the Bayesian approach is due to the fact that it is less liable to the sampling error in consecutive sampling. These results suggest that a carefully designed statistical procedure may reduce the need for the costly generation of large hindcast datasets.

scholarly journals Research of Personalized Recommendation Technology Based on Knowledge Graphs

2021 ◽  
Vol 11 (15) ◽  
pp. 7104
Author(s):  
Xu Yang ◽  
Ziyi Huan ◽  
Yisong Zhai ◽  
Ting Lin
Keyword(s):  
Neural Network ◽  
Hot Spot ◽  
Experimental Results ◽  
Graph Representation ◽  
Personalized Recommendation ◽  
Knowledge Graph ◽  
Learning Technology ◽  
Data Set ◽  
Model Based ◽  
Knowledge Graphs

Nowadays, personalized recommendation based on knowledge graphs has become a hot spot for researchers due to its good recommendation effect. In this paper, we researched personalized recommendation based on knowledge graphs. First of all, we study the knowledge graphs’ construction method and complete the construction of the movie knowledge graphs. Furthermore, we use Neo4j graph database to store the movie data and vividly display it. Then, the classical translation model TransE algorithm in knowledge graph representation learning technology is studied in this paper, and we improved the algorithm through a cross-training method by using the information of the neighboring feature structures of the entities in the knowledge graph. Furthermore, the negative sampling process of TransE algorithm is improved. The experimental results show that the improved TransE model can more accurately vectorize entities and relations. Finally, this paper constructs a recommendation model by combining knowledge graphs with ranking learning and neural network. We propose the Bayesian personalized recommendation model based on knowledge graphs (KG-BPR) and the neural network recommendation model based on knowledge graphs(KG-NN). The semantic information of entities and relations in knowledge graphs is embedded into vector space by using improved TransE method, and we compare the results. The item entity vectors containing external knowledge information are integrated into the BPR model and neural network, respectively, which make up for the lack of knowledge information of the item itself. Finally, the experimental analysis is carried out on MovieLens-1M data set. The experimental results show that the two recommendation models proposed in this paper can effectively improve the accuracy, recall, F1 value and MAP value of recommendation.

Crack-Depth Determination by a Neural Network with a Synthetic Training Data Set

1993 ◽  
pp. 803-810 ◽  
Author(s):  
M. Takadoya ◽  
M. Notake ◽  
M. Kitahara ◽  
J. D. Achenbach ◽  
Q. C. Guo ◽  
...  
Keyword(s):  
Neural Network ◽  
Crack Depth ◽  
Training Data ◽  
Data Set ◽  
Depth Determination ◽  
Synthetic Training Data

scholarly journals Bootstrapping a Neural Morphological Generator from Morphological Analyzer Output for Inuktitut

2019 ◽  
Vol 2 (1) ◽  
Author(s):  
Jeffrey Micher
Keyword(s):  
Neural Network ◽  
Training Data ◽  
Data Set ◽  
Set Size ◽  
The Neural Network ◽  
Surface Character ◽  
Finite State ◽  
Character Sequences ◽  
Finite State Transducer

We present a method for building a morphological generator from the output of an existing analyzer for Inuktitut, in the absence of a two-way finite state transducer which would normally provide this functionality. We make use of a sequence to sequence neural network which “translates” underlying Inuktitut morpheme sequences into surface character sequences. The neural network uses only the previous and the following morphemes as context. We report a morpheme accuracy of approximately 86%. We are able to increase this accuracy slightly by passing deep morphemes directly to output for unknown morphemes. We do not see significant improvement when increasing training data set size, and postulate possible causes for this.

scholarly journals Modeling of daily runoff from a small agricultural watershed using artificial neural network with resampling techniques

2014 ◽  
Vol 17 (1) ◽  
pp. 56-74 ◽  
Author(s):  
Gurjeet Singh ◽  
Rabindra K. Panda ◽  
Marc Lamers
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Agricultural Watershed ◽  
Training Data ◽  
Optimum Number ◽  
Training Process ◽  
Data Set ◽  
Reported Study ◽  
Artificial Neural ◽  
Artificial Neural Network Ann

The reported study was undertaken in a small agricultural watershed, namely, Kapgari in Eastern India having a drainage area of 973 ha. The watershed was subdivided into three sub-watersheds on the basis of drainage network and land topography. An attempt was made to relate the continuously monitored runoff data from the sub-watersheds and the whole-watershed with the rainfall and temperature data using the artificial neural network (ANN) technique. The reported study also evaluated the bias in the prediction of daily runoff with shorter length of training data set using different resampling techniques with the ANN modeling. A 10-fold cross-validation (CV) technique was used to find the optimum number of hidden neurons in the hidden layer and to avoid neural network over-fitting during the training process for shorter length of data. The results illustrated that the ANN models developed with shorter length of training data set avoid neural network over-fitting during the training process, using a 10-fold CV method. Moreover, the biasness was investigated using the bootstrap resampling technique based ANN (BANN) for short length of training data set. In comparison with the 10-fold CV technique, the BANN is more efficient in solving the problems of the over-fitting and under-fitting during training of models for shorter length of data set.

scholarly journals Combined Optimization Prediction Model of Regional Wind Power Based on Convolution Neural Network and Similar Days

Machines ◽  
2020 ◽  
Vol 8 (4) ◽  
pp. 80
Author(s):  
Yalong Li ◽  
Fan Yang ◽  
Wenting Zha ◽  
Licheng Yan
Keyword(s):  
Neural Network ◽  
Prediction Model ◽  
Wind Power ◽  
Prediction Accuracy ◽  
Pearson Correlation ◽  
Combination Method ◽  
Random Fluctuation ◽  
Short Term ◽  
Data Set ◽  
Model Based

With the continuous optimization of energy structures, wind power generation has become the dominant new energy source. The strong random fluctuation of natural wind will bring challenges to power system dispatching, so it is necessary to predict wind power. In order to improve the short-term prediction accuracy of regional wind power, this paper proposes a new combination prediction model based on convolutional neural network (CNN) and similar days analysis. Firstly, the least square fitting and batch normalization (BN) are used to preprocess the data, and then the recent historical wind power data set for CNN is established. Secondly, the Pearson correlation coefficient and cosine similarity combination method are utilized to find similar days in the long-term data set, and the prediction model based on similar days is constructed by the weighting method. Finally, based on the particle swarm optimization (PSO) method, a combined forecasting model is established. The results show that the combined model can accurately predict the future short-term wind power curve, and the prediction accuracy is improved to different extents compared to a single method.

scholarly journals Exploiting the Relationship between Pruning Ratio and Compression Effect for Neural Network Model Based on TensorFlow

2020 ◽  
Vol 2020 ◽  
pp. 1-8
Author(s):  
Bo Liu ◽  
Qilin Wu ◽  
Yiwen Zhang ◽  
Qian Cao
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Prediction Accuracy ◽  
Computing Time ◽  
Experimental Results ◽  
Data Set ◽  
Compression Effect ◽  
Model Based ◽  
The Relationship

Pruning is a method of compressing the size of a neural network model, which affects the accuracy and computing time when the model makes a prediction. In this paper, the hypothesis that the pruning proportion is positively correlated with the compression scale of the model but not with the prediction accuracy and calculation time is put forward. For testing the hypothesis, a group of experiments are designed, and MNIST is used as the data set to train a neural network model based on TensorFlow. Based on this model, pruning experiments are carried out to investigate the relationship between pruning proportion and compression effect. For comparison, six different pruning proportions are set, and the experimental results confirm the above hypothesis.

scholarly journals Fast Linear Adaptive Skipping Training Algorithm for Training Artificial Neural Network

2013 ◽  
Vol 2013 ◽  
pp. 1-9 ◽  
Author(s):  
R. Manjula Devi ◽  
S. Kuppuswami ◽  
R. C. Suganthe
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Training Model ◽  
Training Data ◽  
Experimental Result ◽  
Training Algorithms ◽  
Data Set ◽  
Training Time ◽  
Input Sample ◽  
Artificial Neural

Artificial neural network has been extensively consumed training model for solving pattern recognition tasks. However, training a very huge training data set using complex neural network necessitates excessively high training time. In this correspondence, a new fast Linear Adaptive Skipping Training (LAST) algorithm for training artificial neural network (ANN) is instituted. The core essence of this paper is to ameliorate the training speed of ANN by exhibiting only the input samples that do not categorize perfectly in the previous epoch which dynamically reducing the number of input samples exhibited to the network at every single epoch without affecting the network’s accuracy. Thus decreasing the size of the training set can reduce the training time, thereby ameliorating the training speed. This LAST algorithm also determines how many epochs the particular input sample has to skip depending upon the successful classification of that input sample. This LAST algorithm can be incorporated into any supervised training algorithms. Experimental result shows that the training speed attained by LAST algorithm is preferably higher than that of other conventional training algorithms.

Sign in / Sign up

Export Citation Format

Share Document