scholarly journals A non-exponential extension of Sanov’s theorem via convex duality

2020 ◽  
Vol 52 (1) ◽  
pp. 61-101
Author(s):  
Daniel Lacker
Keyword(s):  
Optimal Transport ◽  
Large Deviation ◽  
Optimization Problems ◽  
Risk Measures ◽  
Approximate Solutions ◽  
Variational Problems ◽  
Dual Pair ◽  
Wasserstein Distance ◽  
Empirical Measures ◽  
Empirical Distributions

AbstractThis work is devoted to a vast extension of Sanov’s theorem, in Laplace principle form, based on alternatives to the classical convex dual pair of relative entropy and cumulant generating functional. The abstract results give rise to a number of probabilistic limit theorems and asymptotics. For instance, widely applicable non-exponential large deviation upper bounds are derived for empirical distributions and averages of independent and identically distributed samples under minimal integrability assumptions, notably accommodating heavy-tailed distributions. Other interesting manifestations of the abstract results include new results on the rate of convergence of empirical measures in Wasserstein distance, uniform large deviation bounds, and variational problems involving optimal transport costs, as well as an application to error estimates for approximate solutions of stochastic optimization problems. The proofs build on the Dupuis–Ellis weak convergence approach to large deviations as well as the duality theory for convex risk measures.

scholarly journals Extended Laplace principle for empirical measures of a Markov chain

2019 ◽  
Vol 51 (01) ◽  
pp. 136-167 ◽  
Author(s):  
Stephan Eckstein
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Large Deviations ◽  
Large Deviation ◽  
Dual Pair ◽  
Wasserstein Distance ◽  
Large Deviations Principle ◽  
Empirical Measures ◽  
Leibler Divergence ◽  
Laplace Principle

AbstractWe consider discrete-time Markov chains with Polish state space. The large deviations principle for empirical measures of a Markov chain can equivalently be stated in Laplace principle form, which builds on the convex dual pair of relative entropy (or Kullback– Leibler divergence) and cumulant generating functional f ↦ ln ʃ exp (f). Following the approach by Lacker (2016) in the independent and identically distributed case, we generalize the Laplace principle to a greater class of convex dual pairs. We present in depth one application arising from this extension, which includes large deviation results and a weak law of large numbers for certain robust Markov chains—similar to Markov set chains—where we model robustness via the first Wasserstein distance. The setting and proof of the extended Laplace principle are based on the weak convergence approach to large deviations by Dupuis and Ellis (2011).

scholarly journals Transfer Learning for Wireless Fingerprinting Localization Based on Optimal Transport

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6994
Author(s):  
Siqi Bai ◽  
Yongjie Luo ◽  
Qun Wan
Keyword(s):  
Transfer Learning ◽  
Optimal Transport ◽  
Wasserstein Distance ◽  
Target Domain ◽  
The Public ◽  
Empirical Distributions ◽  
Feature Based ◽  
And Performance ◽  
The Cost ◽  
Radio Map

Wireless fingerprinting localization (FL) systems identify locations by building radio fingerprint maps, aiming to provide satisfactory location solutions for the complex environment. However, the radio map is easy to change, and the cost of building a new one is high. One research focus is to transfer knowledge from the old radio maps to a new one. Feature-based transfer learning methods help by mapping the source fingerprint and the target fingerprint to a common hidden domain, then minimize the maximum mean difference (MMD) distance between the empirical distributions in the latent domain. In this paper, the optimal transport (OT)-based transfer learning is adopted to directly map the fingerprint from the source domain to the target domain by minimizing the Wasserstein distance so that the data distribution of the two domains can be better matched and the positioning performance in the target domain is improved. Two channel-models are used to simulate the transfer scenarios, and the public measured data test further verifies that the transfer learning based on OT has better accuracy and performance when the radio map changes in FL, indicating the importance of the method in this field.

scholarly journals Stratification and Optimal Resampling for Sequential Monte Carlo

Biometrika ◽  
2021 ◽  
Author(s):  
Yichao Li ◽  
Wenshuo Wang ◽  
Ke Deng ◽  
Jun S Liu
Keyword(s):  
Monte Carlo ◽  
Sequential Monte Carlo ◽  
Optimal Transport ◽  
Wasserstein Distance ◽  
Hilbert Curve ◽  
Quasi Monte Carlo ◽  
Empirical Distributions ◽  
Monte Carlo Algorithms ◽  
Energy Distance ◽  
Optimal Resampling

Abstract Sequential Monte Carlo algorithms have been widely accepted as a powerful computational tool for making inference with dynamical systems. A key step in sequential Monte Carlo is resampling, which plays a role of steering the algorithm towards the future dynamics. Several strategies have been used in practice, including multinomial resampling, residual resampling, optimal resampling, stratified resampling, and optimal transport resampling. In the one-dimensional cases, we show that optimal transport resampling is equivalent to stratified resampling on the sorted particles, and they both minimize the resampling variance as well as the expected squared energy distance between the original and resampled empirical distributions. In general d-dimensional cases, if the particles are first sorted using the Hilbert curve, we show that the variance of stratified resampling is O(m-(1+2/d)), an improved rate compared to the previously known best rate O(m-(1+1/d)), where m is the number of resampled particles. We show this improved rate is optimal for ordered stratified resampling schemes, as conjectured in Gerber et al. (2019).We also present an almost sure bound on the Wasserstein distance between the original and Hilbert-curve-resampled empirical distributions. In light of these results, we show that, for dimension d > 1, the mean square error of sequential quasi-Monte Carlo with n particles can be O(n-1-4/{d(d+4)}) if Hilbert curve resampling is used and a specific low-discrepancy set is chosen. To our knowledge, this is the first known convergence rate lower than o(n-1).

279 WACSAW: A Statistical and Individually Adaptive Method to Determine Sleep and Wakefulness from Activity

SLEEP ◽  
2021 ◽  
Vol 44 (Supplement_2) ◽  
pp. A111-A112
Author(s):  
Austin Vandegriffe ◽  
V A Samaranayake ◽  
Matthew Thimgan
Keyword(s):  
Time Series ◽  
Optimal Transport ◽  
Activity Patterns ◽  
Adaptive Method ◽  
Wasserstein Distance ◽  
Activity Data ◽  
Commercial Grade ◽  
Movement Data ◽  
The Difference ◽  
Levene Test

Abstract Introduction Technological innovations have broadened the type and amount of activity data that can be captured in the home and under normal living conditions. Yet, converting naturalistic activity patterns into sleep and wakefulness states has remained a challenge. Despite the successes of current algorithms, they do not fill all actigraphy needs. We have developed a novel statistical approach to determine sleep and wakefulness times, called the Wasserstein Algorithm for Classifying Sleep and Wakefulness (WACSAW), and validated the algorithm in a small cohort of healthy participants. Methods WACSAW functional routines: 1) Conversion of the triaxial movement data into a univariate time series; 2) Construction of a Wasserstein weighted sum (WSS) time series by measuring the Wasserstein distance between equidistant distributions of movement data before and after the time-point of interest; 3) Segmenting the time series by identifying changepoints based on the behavior of the WSS series; 4) Merging segments deemed similar by the Levene test; 5) Comparing segments by optimal transport methodology to determine the difference from a flat, invariant distribution at zero. The resulting histogram can be used to determine sleep and wakefulness parameters around a threshold determined for each individual based on histogram properties. To validate the algorithm, participants wore the GENEActiv and a commercial grade actigraphy watch for 48 hours. The accuracy of WACSAW was compared to a detailed activity log and benchmarked against the results of the output from commercial wrist actigraph. Results WACSAW performed with an average accuracy, sensitivity, and specificity of >95% compared to detailed activity logs in 10 healthy-sleeping individuals of mixed sexes and ages. We then compared WACSAW’s performance against a common wrist-worn, commercial sleep monitor. WACSAW outperformed the commercial grade system in each participant compared to activity logs and the variability between subjects was cut substantially. Conclusion The performance of WACSAW demonstrates good results in a small test cohort. In addition, WACSAW is 1) open-source, 2) individually adaptive, 3) indicates individual reliability, 4) based on the activity data stream, and 5) requires little human intervention. WACSAW is worthy of validating against polysomnography and in patients with sleep disorders to determine its overall effectiveness. Support (if any):

scholarly journals Gromov-Wasserstein optimal transport to align single-cell multi-omics data

2020 ◽  
Author(s):  
Pinar Demetci ◽  
Rebecca Santorella ◽  
Björn Sandstede ◽  
William Stafford Noble ◽  
Ritambhara Singh
Keyword(s):  
Single Cell ◽  
Optimal Transport ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Single Cells ◽  
Wasserstein Distance ◽  
Cell Alignment ◽  
Shared Space ◽  
Real World Datasets ◽  
Unsupervised Algorithms

AbstractData integration of single-cell measurements is critical for understanding cell development and disease, but the lack of correspondence between different types of measurements makes such efforts challenging. Several unsupervised algorithms can align heterogeneous single-cell measurements in a shared space, enabling the creation of mappings between single cells in different data domains. However, these algorithms require hyperparameter tuning for high-quality alignments, which is difficult in an unsupervised setting without correspondence information for validation. We present Single-Cell alignment using Optimal Transport (SCOT), an unsupervised learning algorithm that uses Gromov Wasserstein-based optimal transport to align single-cell multi-omics datasets. We compare the alignment performance of SCOT with state-of-the-art algorithms on four simulated and two real-world datasets. SCOT performs on par with state-of-the-art methods but is faster and requires tuning fewer hyperparameters. Furthermore, we provide an algorithm for SCOT to use Gromov Wasserstein distance to guide the parameter selection. Thus, unlike previous methods, SCOT aligns well without using any orthogonal correspondence information to pick the hyperparameters. Our source code and scripts for replicating the results are available at https://github.com/rsinghlab/SCOT.

scholarly journals A Dai-Liao Hybrid Hestenes-Stiefel and Fletcher-Revees Methods for Unconstrained Optimization

2021 ◽  
Vol 2 (1) ◽  
pp. 33
Author(s):  
Nasiru Salihu ◽  
Mathew Remilekun Odekunle ◽  
Also Mohammed Saleh ◽  
Suraj Salihu
Keyword(s):  
Unconstrained Optimization ◽  
Line Search ◽  
Optimization Problems ◽  
Convex Combination ◽  
Gradient Methods ◽  
Approximate Solutions ◽  
Step Size ◽  
Conjugacy Condition ◽  
Inexact Line Search ◽  
Wolfe Line Search

Some problems have no analytical solution or too difficult to solve by scientists, engineers, and mathematicians, so the development of numerical methods to obtain approximate solutions became necessary. Gradient methods are more efficient when the function to be minimized continuously in its first derivative. Therefore, this article presents a new hybrid Conjugate Gradient (CG) method to solve unconstrained optimization problems. The method requires the first-order derivatives but overcomes the steepest descent method’s shortcoming of slow convergence and needs not to save or compute the second-order derivatives needed by the Newton method. The CG update parameter is suggested from the Dai-Liao conjugacy condition as a convex combination of Hestenes-Stiefel and Fletcher-Revees algorithms by employing an optimal modulating choice parameterto avoid matrix storage. Numerical computation adopts an inexact line search to obtain the step-size that generates a decent property, showing that the algorithm is robust and efficient. The scheme converges globally under Wolfe line search, and it’s like is suitable in compressive sensing problems and M-tensor systems.

scholarly journals Robustness in the Optimization of Risk Measures

2021 ◽  
Author(s):  
Paul Embrechts ◽  
Alexander Schied ◽  
Ruodu Wang
Keyword(s):  
Value At Risk ◽  
Optimization Problem ◽  
Financial Regulation ◽  
Optimization Problems ◽  
Risk Measures ◽  
Risk Measurement ◽  
Loss Functions ◽  
Insurance Regulation ◽  
Convex Risk Measures ◽  
Comparative Advantages

We study issues of robustness in the context of Quantitative Risk Management and Optimization. We develop a general methodology for determining whether a given risk-measurement-related optimization problem is robust, which we call “robustness against optimization.” The new notion is studied for various classes of risk measures and expected utility and loss functions. Motivated by practical issues from financial regulation, special attention is given to the two most widely used risk measures in the industry, Value-at-Risk (VaR) and Expected Shortfall (ES). We establish that for a class of general optimization problems, VaR leads to nonrobust optimizers, whereas convex risk measures generally lead to robust ones. Our results offer extra insight on the ongoing discussion about the comparative advantages of VaR and ES in banking and insurance regulation. Our notion of robustness is conceptually different from the field of robust optimization, to which some interesting links are derived.

On the computation of measure-valued solutions

Acta Numerica ◽  
2016 ◽  
Vol 25 ◽  
pp. 567-679 ◽  
Author(s):  
Ulrik S. Fjordholm ◽  
Siddhartha Mishra ◽  
Eitan Tadmor
Keyword(s):  
Fluid Dynamics ◽  
Materials Science ◽  
Numerical Algorithms ◽  
Approximate Solutions ◽  
Variational Problems ◽  
Convincing Evidence ◽  
Young Measures ◽  
Inviscid Fluid ◽  
New Paradigm ◽  
Uniqueness Of Solutions

A standard paradigm for the existence of solutions in fluid dynamics is based on the construction of sequences of approximate solutions or approximate minimizers. This approach faces serious obstacles, most notably in multi-dimensional problems, where the persistence of oscillations at ever finer scales prevents compactness. Indeed, these oscillations are an indication, consistent with recent theoretical results, of the possible lack of existence/uniqueness of solutions within the standard framework of integrable functions. It is in this context that Young measures – parametrized probability measures which can describe the limits of such oscillatory sequences – offer the more general paradigm of measure-valued solutions for these problems.We present viable numerical algorithms to compute approximate measure-valued solutions, based on the realization of approximate measures as laws of Monte Carlo sampled random fields. We prove convergence of these algorithms to measure-valued solutions for the equations of compressible and incompressible inviscid fluid dynamics, and present a large number of numerical experiments which provide convincing evidence for the viability of the new paradigm. We also discuss the use of these algorithms, and their extensions, in uncertainty quantification and contexts other than fluid dynamics, such as non-convex variational problems in materials science.

Sign in / Sign up

Export Citation Format

Share Document