scholarly journals New Estimators of the Bayes Factor for Models with High-Dimensional Parameter and/or Latent Variable Spaces

Entropy ◽  
2021 ◽  
Vol 23 (4) ◽  
pp. 399
Author(s):  
Anna Pajor
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Bayes Factor ◽  
High Dimensional ◽  
Model Parameters ◽  
Posterior Odds ◽  
Dimensional Parameter ◽  
Data Set ◽  
Multivariate Stochastic Volatility ◽  
Competing Models

Formal Bayesian comparison of two competing models, based on the posterior odds ratio, amounts to estimation of the Bayes factor, which is equal to the ratio of respective two marginal data density values. In models with a large number of parameters and/or latent variables, they are expressed by high-dimensional integrals, which are often computationally infeasible. Therefore, other methods of evaluation of the Bayes factor are needed. In this paper, a new method of estimation of the Bayes factor is proposed. Simulation examples confirm good performance of the proposed estimators. Finally, these new estimators are used to formally compare different hybrid Multivariate Stochastic Volatility–Multivariate Generalized Autoregressive Conditional Heteroskedasticity (MSV-MGARCH) models which have a large number of latent variables. The empirical results show, among other things, that the validity of reduction of the hybrid MSV-MGARCH model to the MGARCH specification depends on the analyzed data set as well as on prior assumptions about model parameters.

scholarly journals Gaussian Processes Proxy Model with Latent Variable Models and Variogram-Based Sensitivity Analysis for Assisted History Matching

Energies ◽  
2020 ◽  
Vol 13 (17) ◽  
pp. 4290
Author(s):  
Dongmei Zhang ◽  
Yuyang Zhang ◽  
Bohou Jiang ◽  
Xinwei Jiang ◽  
Zhijiang Kang
Keyword(s):  
Sensitivity Analysis ◽  
Gaussian Processes ◽  
Latent Variable ◽  
History Matching ◽  
Latent Variable Models ◽  
High Dimensional ◽  
Model Parameters ◽  
Variable Model ◽  
Assisted History Matching ◽  
Proxy Models

Reservoir history matching is a well-known inverse problem for production prediction where enormous uncertain reservoir parameters of a reservoir numerical model are optimized by minimizing the misfit between the simulated and history production data. Gaussian Process (GP) has shown promising performance for assisted history matching due to the efficient nonparametric and nonlinear model with few model parameters to be tuned automatically. Recently introduced Gaussian Processes proxy models and Variogram Analysis of Response Surface-based sensitivity analysis (GP-VARS) uses forward and inverse Gaussian Processes (GP) based proxy models with the VARS-based sensitivity analysis to optimize the high-dimensional reservoir parameters. However, the inverse GP solution (GPIS) in GP-VARS are unsatisfactory especially for enormous reservoir parameters where the mapping from low-dimensional misfits to high-dimensional uncertain reservoir parameters could be poorly modeled by GP. To improve the performance of GP-VARS, in this paper we propose the Gaussian Processes proxy models with Latent Variable Models and VARS-based sensitivity analysis (GPLVM-VARS) where Gaussian Processes Latent Variable Model (GPLVM)-based inverse solution (GPLVMIS) instead of GP-based GPIS is provided with the inputs and outputs of GPIS reversed. The experimental results demonstrate the effectiveness of the proposed GPLVM-VARS in terms of accuracy and complexity. The source code of the proposed GPLVM-VARS is available at https://github.com/XinweiJiang/GPLVM-VARS.

A GAMMA MOVING AVERAGE PROCESS FOR MODELLING DEPENDENCE ACROSS DEVELOPMENT YEARS IN RUN-OFF TRIANGLES

2020 ◽  
pp. 1-22
Author(s):  
Luis E. Nieto-Barajas ◽  
Rodrigo S. Targino
Keyword(s):  
Latent Variables ◽  
Moving Average ◽  
Real Data ◽  
Model Parameters ◽  
Average Process ◽  
Moving Average Process ◽  
Data Set ◽  
Run Off ◽  
Average Form ◽  
Reserve Estimates

ABSTRACT We propose a stochastic model for claims reserving that captures dependence along development years within a single triangle. This dependence is based on a gamma process with a moving average form of order $p \ge 0$ which is achieved through the use of poisson latent variables. We carry out Bayesian inference on model parameters and borrow strength across several triangles, coming from different lines of businesses or companies, through the use of hierarchical priors. We carry out a simulation study as well as a real data analysis. Results show that reserve estimates, for the real data set studied, are more accurate with our gamma dependence model as compared to the benchmark over-dispersed poisson that assumes independence.

Towards an Efficient Validation of Dynamical Whole-brain Models

2021 ◽  
Author(s):  
Kevin J. Wischnewski ◽  
Simon B. Eickhoff ◽  
Viktor K. Jirsa ◽  
Oleksandr V. Popovych
Keyword(s):  
Structural Connectivity ◽  
Three Dimensional ◽  
Bayesian Optimization ◽  
High Dimensional ◽  
Brain Dynamics ◽  
Model Parameters ◽  
Phase Oscillators ◽  
Dimensional Parameter ◽  
Whole Brain ◽  
Brain Models

Abstract Simulating the resting-state brain dynamics via mathematical whole-brain models requires an optimal selection of parameters, which determine the model’s capability to replicate empirical data. Since the parameter optimization via a grid search (GS) becomes unfeasible for high-dimensional models, we evaluate several alternative approaches to maximize the correspondence between simulated and empirical functional connectivity. A dense GS serves as a benchmark to assess the performance of four optimization schemes: Nelder-Mead Algorithm (NMA), Particle Swarm Optimization (PSO), Covariance Matrix Adaptation Evolution Strategy (CMAES) and Bayesian Optimization (BO). To compare them, we employ an ensemble of coupled phase oscillators built upon individual empirical structural connectivity of 105 healthy subjects. We determine optimal model parameters from two- and three-dimensional parameter spaces and show that the overall fitting quality of the tested methods can compete with the GS. There are, however, marked differences in the required computational resources and stability properties, which we also investigate before proposing CMAES and BO as efficient alternatives to a high-dimensional GS. For the three-dimensional case, these methods generated similar results as the GS, but within less than 6% of the computation time. Our results contribute to an efficient validation of models for personalized simulations of brain dynamics.

scholarly journals Variational Inference over Nonstationary Data Streams for Exponential Family Models

Mathematics ◽  
2020 ◽  
Vol 8 (11) ◽  
pp. 1942
Author(s):  
Andrés R. Masegosa ◽  
Darío Ramos-López ◽  
Antonio Salmerón ◽  
Helge Langseth ◽  
Thomas D. Nielsen
Keyword(s):  
Bayesian Inference ◽  
Variational Methods ◽  
Latent Variables ◽  
Exponential Family ◽  
Concept Drift ◽  
Variational Inference ◽  
Model Parameters ◽  
Time Step ◽  
Data Set ◽  
Family Models

In many modern data analysis problems, the available data is not static but, instead, comes in a streaming fashion. Performing Bayesian inference on a data stream is challenging for several reasons. First, it requires continuous model updating and the ability to handle a posterior distribution conditioned on an unbounded data set. Secondly, the underlying data distribution may drift from one time step to another, and the classic i.i.d. (independent and identically distributed), or data exchangeability assumption does not hold anymore. In this paper, we present an approximate Bayesian inference approach using variational methods that addresses these issues for conjugate exponential family models with latent variables. Our proposal makes use of a novel scheme based on hierarchical priors to explicitly model temporal changes of the model parameters. We show how this approach induces an exponential forgetting mechanism with adaptive forgetting rates. The method is able to capture the smoothness of the concept drift, ranging from no drift to abrupt drift. The proposed variational inference scheme maintains the computational efficiency of variational methods over conjugate models, which is critical in streaming settings. The approach is validated on four different domains (energy, finance, geolocation, and text) using four real-world data sets.

scholarly journals Efficient ensemble generation for uncertain correlated parameters in atmospheric chemical models

2021 ◽  
Author(s):  
Annika Vogel ◽  
Hendrik Elbern
Keyword(s):  
Forecast Model ◽  
High Dimensional ◽  
Biogenic Emissions ◽  
Model Parameters ◽  
Emission Rates ◽  
Model Uncertainties ◽  
Dimensional Parameter ◽  
Novel Approach ◽  
Correlated Parameters ◽  
Ensemble Generation

Abstract. Atmospheric chemical forecasts highly rely on various model parameters, which are often insufficiently known, as emission rates and deposition velocities. However, a reliable estimation of resulting uncertainties by an ensemble of forecasts is impaired by the high-dimensionality of the system. This study presents a novel approach to efficiently perturb atmospheric-chemical model parameters according to their leading coupled uncertainties. The algorithm is based on the idea that the forecast model acts as a dynamical system inducing multi-variational correlations of model uncertainties. The specific algorithm presented in this study is designed for parameters which depend on local environmental conditions and consists of three major steps: (1) an efficient assessment of various sources of model uncertainties spanned by independent sensitivities, (2) an efficient extraction of leading coupled uncertainties using eigenmode decomposition, and (3) an efficient generation of perturbations for high-dimensional parameter fields by the Karhunen-Loéve expansion. Due to their perceived simulation challenge the method has been applied to biogenic emissions of five trace gases, considering state-dependent sensitivities to local atmospheric and terrestrial conditions. Rapidly decreasing eigenvalues state high spatial- and cross-correlations of regional biogenic emissions, which are represented by a low number of dominating components. Consequently, leading uncertainties can be covered by low number of perturbations enabling ensemble sizes of the order of 10 members. This demonstrates the suitability of the algorithm for efficient ensemble generation for high-dimensional atmospheric chemical parameters.

scholarly journals Regularized Structural Equation Modeling with Stability Selection

2020 ◽  
Author(s):  
Xiaobei Li ◽  
Ross Jacobucci
Keyword(s):  
Structural Equation Modeling ◽  
Latent Variables ◽  
Latent Variable ◽  
Structural Equation ◽  
Equation Modeling ◽  
High Dimensional ◽  
Free Lunch ◽  
Stability Selection ◽  
Selection Operator ◽  
No Free Lunch

Regularization methods such as the least absolute shrinkage and selection operator (LASSO) are commonly used in high dimensional data to achieve sparser solutions. They are also becoming increasingly popular in social and behavioral research. Recently methods such as regularized structural equation modeling (SEM) and penalized likelihood SEM have been proposed, trying to transfer the benefits of regularization to models with latent variables involved. However, some drawbacks of the LASSO such as high false positive rates (FPRs) and inconsistency in selection results persist at the same time. We propose the use of stability selection (Meinshausen & Bu ̈hlmann, 2010) as a mechanism to overcome these limitations, demonstrating simulation conditions in which it improves performance, and simulation conditions in which it does not. In this paper, we point out that there is no free lunch, and researchers should be aware of those problems when applying regularization to latent variable models, concluding with an empirical example and further discussion of the application of regularization to SEM.

scholarly journals Item response theory modeling for microarray gene expression data

2009 ◽  
Vol 6 (1) ◽  
Author(s):  
Andrej Kastrin
Keyword(s):  
Gene Expression ◽  
Item Response ◽  
Microarray Data ◽  
Latent Variables ◽  
Latent Variable ◽  
Expression Profiles ◽  
Gene Expression Profiles ◽  
High Dimensional ◽  
Variable Model ◽  
Cluster Partition

The high dimensionality of global gene expression profiles, where number of variables (genes) is very large compared to the number of observations (samples), presents challenges that affect generalizability and applicability of microarray analysis. Latent variable modeling offers a promising approach to deal with high-dimensional microarray data. The latent variable model is based on a few latent variables that capture most of the gene expression information. Here, we describe how to accomplish a reduction in dimension by a latent variable methodology, which can greatly reduce the number of features used to characterize microarray data. We propose a general latent variable framework for prediction of predefined classes of samples using gene expression profiles from microarray experiments. The framework consists of (i) selection of smaller number of genes that are most differentially expressed between samples, (ii) dimension reduction using hierarchical clustering, where each cluster partition is identified as latent variable, (iii) discretization of gene expression matrix, (iv) fitting the Rasch item response model for genes in each cluster partition to estimate the expression of latent variable, and (v) construction of prediction model with latent variables as covariates to study the relationship between latent variables and phenotype. Two different microarray data sets are used to illustrate a general framework of the approach. We show that the predictive performance of our method is comparable to the current best approach based on an all-gene space. The method is general and can be applied to the other high-dimensional data problems.

scholarly journals Constraining the parameters of high-dimensional models with active learning

2019 ◽  
Vol 79 (11) ◽  
Author(s):  
Sascha Caron ◽  
Tom Heskes ◽  
Sydney Otten ◽  
Bob Stienen
Keyword(s):  
Active Learning ◽  
Particle Physics ◽  
High Energy Physics ◽  
High Energy ◽  
Physical Models ◽  
High Dimensional ◽  
Physical Parameters ◽  
Model Parameters ◽  
Dimensional Parameter ◽  
Parameter Spaces

AbstractConstraining the parameters of physical models with $$>5-10$$>5-10 parameters is a widespread problem in fields like particle physics and astronomy. The generation of data to explore this parameter space often requires large amounts of computational resources. The commonly used solution of reducing the number of relevant physical parameters hampers the generality of the results. In this paper we show that this problem can be alleviated by the use of active learning. We illustrate this with examples from high energy physics, a field where simulations are often expensive and parameter spaces are high-dimensional. We show that the active learning techniques query-by-committee and query-by-dropout-committee allow for the identification of model points in interesting regions of high-dimensional parameter spaces (e.g. around decision boundaries). This makes it possible to constrain model parameters more efficiently than is currently done with the most common sampling algorithms and to train better performing machine learning models on the same amount of data. Code implementing the experiments in this paper can be found on GitHub "Image missing"

scholarly journals Revealing unobserved factors underlying cortical activity using a rectified latent variable model applied to neural population recordings

2016 ◽  
Author(s):  
Matthew R. Whiteway ◽  
Daniel A. Butts
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Large Population ◽  
Component Analysis ◽  
Latent Variable Model ◽  
Neural Population ◽  
Model Parameters ◽  
Variable Model ◽  
Sensory Stimuli ◽  
External Stimuli

ABSTRACTThe activity of sensory cortical neurons is not only driven by external stimuli, but is also shaped by other sources of input to the cortex. Unlike external stimuli these other sources of input are challenging to experimentally control or even observe, and as a result contribute to variability of neuronal responses to sensory stimuli. However, such sources of input are likely not “noise”, and likely play an integral role in sensory cortex function. Here, we introduce the rectified latent variable model (RLVM) in order to identify these sources of input using simultaneously recorded cortical neuron populations. The RLVM is novel in that it employs non-negative (rectified) latent variables, and is able to be much less restrictive in the mathematical constraints on solutions due to the use an autoencoder neural network to initialize model parameters. We show the RLVM outperforms principal component analysis, factor analysis and independent component analysis across a variety of measures using simulated data. We then apply this model to the 2-photon imaging of hundreds of simultaneously recorded neurons in mouse primary somatosensory cortex during a tactile discrimination task. Across many experiments, the RLVM identifies latent variables related to both the tactile stimulation as well as non-stimulus aspects of the behavioral task, with a majority of activity explained by the latter. These results suggest that properly identifying such latent variables is necessary for a full understanding of sensory cortical function, and demonstrates novel methods for leveraging large population recordings to this end.

scholarly journals Revealing unobserved factors underlying cortical activity with a rectified latent variable model applied to neural population recordings

2017 ◽  
Vol 117 (3) ◽  
pp. 919-936 ◽  
Author(s):  
Matthew R. Whiteway ◽  
Daniel A. Butts
Keyword(s):  
Latent Variables ◽  
Latent Variable ◽  
Large Population ◽  
Component Analysis ◽  
Latent Variable Model ◽  
Neural Population ◽  
Model Parameters ◽  
Variable Model ◽  
Sensory Stimuli ◽  
External Stimuli

The activity of sensory cortical neurons is not only driven by external stimuli but also shaped by other sources of input to the cortex. Unlike external stimuli, these other sources of input are challenging to experimentally control, or even observe, and as a result contribute to variability of neural responses to sensory stimuli. However, such sources of input are likely not “noise” and may play an integral role in sensory cortex function. Here we introduce the rectified latent variable model (RLVM) in order to identify these sources of input using simultaneously recorded cortical neuron populations. The RLVM is novel in that it employs nonnegative (rectified) latent variables and is much less restrictive in the mathematical constraints on solutions because of the use of an autoencoder neural network to initialize model parameters. We show that the RLVM outperforms principal component analysis, factor analysis, and independent component analysis, using simulated data across a range of conditions. We then apply this model to two-photon imaging of hundreds of simultaneously recorded neurons in mouse primary somatosensory cortex during a tactile discrimination task. Across many experiments, the RLVM identifies latent variables related to both the tactile stimulation as well as nonstimulus aspects of the behavioral task, with a majority of activity explained by the latter. These results suggest that properly identifying such latent variables is necessary for a full understanding of sensory cortical function and demonstrate novel methods for leveraging large population recordings to this end. NEW & NOTEWORTHY The rapid development of neural recording technologies presents new opportunities for understanding patterns of activity across neural populations. Here we show how a latent variable model with appropriate nonlinear form can be used to identify sources of input to a neural population and infer their time courses. Furthermore, we demonstrate how these sources are related to behavioral contexts outside of direct experimental control.

Sign in / Sign up

Export Citation Format

Share Document