scholarly journals An adaptive Hessian approximated stochastic gradient MCMC method

2021 ◽  
Vol 432 ◽  
pp. 110150
Author(s):  
Yating Wang ◽  
Wei Deng ◽  
Guang Lin
Keyword(s):  
Stochastic Gradient ◽  
Mcmc Method

scholarly journals Improving sampling accuracy of stochastic gradient MCMC methods via non-uniform subsampling of gradients

2021 ◽  
Vol 0 (0) ◽  
pp. 0
Author(s):  
Ruilin Li ◽  
Xin Wang ◽  
Hongyuan Zha ◽  
Molei Tao
Keyword(s):  
Large Scale ◽  
Variance Reduction ◽  
Sampling Error ◽  
Stochastic Gradient ◽  
Mcmc Methods ◽  
Practical Implementation ◽  
Mcmc Method ◽  
Local Variance ◽  
Gradient Based ◽  
Data Points

<p style='text-indent:20px;'>Many Markov Chain Monte Carlo (MCMC) methods leverage gradient information of the potential function of target distribution to explore sample space efficiently. However, computing gradients can often be computationally expensive for large scale applications, such as those in contemporary machine learning. Stochastic Gradient (SG-)MCMC methods approximate gradients by stochastic ones, commonly via uniformly subsampled data points, and achieve improved computational efficiency, however at the price of introducing sampling error. We propose a non-uniform subsampling scheme to improve the sampling accuracy. The proposed exponentially weighted stochastic gradient (EWSG) is designed so that a non-uniform-SG-MCMC method mimics the statistical behavior of a batch-gradient-MCMC method, and hence the inaccuracy due to SG approximation is reduced. EWSG differs from classical variance reduction (VR) techniques as it focuses on the entire distribution instead of just the variance; nevertheless, its reduced local variance is also proved. EWSG can also be viewed as an extension of the importance sampling idea, successful for stochastic-gradient-based optimizations, to sampling tasks. In our practical implementation of EWSG, the non-uniform subsampling is performed efficiently via a Metropolis-Hastings chain on the data index, which is coupled to the MCMC algorithm. Numerical experiments are provided, not only to demonstrate EWSG's effectiveness, but also to guide hyperparameter choices, and validate our <i>non-asymptotic global error bound</i> despite of approximations in the implementation. Notably, while statistical accuracy is improved, convergence speed can be comparable to the uniform version, which renders EWSG a practical alternative to VR (but EWSG and VR can be combined too).</p>

scholarly journals Semi-supervised Max-margin Topic Model with Manifold Posterior Regularization

2017 ◽  
Author(s):  
Wenbo Hu ◽  
Jun Zhu ◽  
Hang Su ◽  
Jingwei Zhuo ◽  
Bo Zhang
Keyword(s):  
Supervised Learning ◽  
Topic Model ◽  
Topic Models ◽  
Stochastic Gradient ◽  
Mcmc Method ◽  
Tight Coupling ◽  
Label Information ◽  
Latent Topic ◽  
Latent Topics ◽  
Qualitative Performance

Supervised topic models leverage label information to learn discriminative latent topic representations. As collecting a fully labeled dataset is often time-consuming, semi-supervised learning is of high interest. In this paper, we present an effective semi-supervised max-margin topic model by naturally introducing manifold posterior regularization to a regularized Bayesian topic model, named LapMedLDA. The model jointly learns latent topics and a related classifier with only a small fraction of labeled documents. To perform the approximate inference, we derive an efficient stochastic gradient MCMC method. Unlike the previous semi-supervised topic models, our model adopts a tight coupling between the generative topic model and the discriminative classifier. Extensive experiments demonstrate that such tight coupling brings significant benefits in quantitative and qualitative performance.

Share Market Data Prediction Strategies using Deep Learning Algorithm

2019 ◽  
Vol 13 ◽  
Author(s):  
A John. ◽  
D. Praveen Dominic ◽  
M. Adimoolam ◽  
N. M. Balamurugan
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Stock Market ◽  
Predictive Analytics ◽  
Learning Algorithm ◽  
Market Price ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Mining Machine ◽  
Gradient Descent Algorithm

Background:: Predictive analytics has a multiplicity of statistical schemes from predictive modelling, data mining, machine learning. It scrutinizes present and chronological data to make predictions about expectations or if not unexplained measures. Most predictive models are used for business analytics to overcome loses and profit gaining. Predictive analytics is used to exploit the pattern in old and historical data. Objective: People used to follow some strategies for predicting stock value to invest in the more profit-gaining stocks and those strategies to search the stock market prices which are incorporated in some intelligent methods and tools. Such strategies will increase the investor’s profits and also minimize their risks. So prediction plays a vital role in stock market gaining and is also a very intricate and challenging process. Method: The proposed optimized strategies are the Deep Neural Network with Stochastic Gradient for stock prediction. The Neural Network is trained using Back-propagation neural networks algorithm and stochastic gradient descent algorithm as optimal strategies. Results: The experiment is conducted for stock market price prediction using python language with the visual package. In this experiment RELIANCE.NS, TATAMOTORS.NS, and TATAGLOBAL.NS dataset are taken as input dataset and it is downloaded from National Stock Exchange site. The artificial neural network component including Deep Learning model is most effective for more than 100,000 data points to train this model. This proposed model is developed on daily prices of stock market price to understand how to build model with better performance than existing national exchange method.

Sign in / Sign up

Export Citation Format

Share Document