A Stochastic Derivative-Free Optimization Method with Importance Sampling: Theory and Learning to Control

We consider the problem of unconstrained minimization of a smooth objective function in ℝn in a setting where only function evaluations are possible. While importance sampling is one of the most popular techniques used by machine learning practitioners to accelerate the convergence of their models when applicable, there is not much existing theory for this acceleration in the derivative-free setting. In this paper, we propose the first derivative free optimization method with importance sampling and derive new improved complexity results on non-convex, convex and strongly convex functions. We conduct extensive experiments on various synthetic and real LIBSVM datasets confirming our theoretical results. We test our method on a collection of continuous control tasks on MuJoCo environments with varying difficulty. Experiments show that our algorithm is practical for high dimensional continuous control problems where importance sampling results in a significant sample complexity improvement.

Operators of stochastic differentiation on spaces of nonregular generalized functions of Levy white noise analysis

The operators of stochastic differentiation, which are closely related with the extended Skorohod stochastic integral and with the Hida stochastic derivative, play an important role in the classical (Gaussian) white noise analysis. In particular, these operators can be used in order to study some properties of the extended stochastic integral and of solutions of stochastic equations with Wick-type nonlinearities. During recent years the operators of stochastic differentiation were introduced and studied, in particular, in the framework of the Meixner white noise analysis, in the same way as on spaces of regular test and generalized functions and on spaces of nonregular test functions of the Levy white noise analysis. In the present paper we make the next natural step: introduce and study operators of stochastic differentiation on spaces of nonregular generalized functions of the Levy white noise analysis (i.e., on spaces of generalized functions that belong to the so-called nonregular rigging of the space of square integrable with respect to the measure of a Levy white noise functions). In so doing, we use Lytvynov's generalization of the chaotic representation property. The researches of the present paper can be considered as a contribution in a further development of the Levy white noise analysis.

On operators of stochastic differentiation on spaces of regular test and generalized functions of Lévy white noise analysis

The operators of stochastic differentiation, which are closely related with the extended Skorohod stochastic integral and with the Hida stochastic derivative, play an important role in the classical (Gaussian) white noise analysis. In particular, these operators can be used in order to study properties of the extended stochastic integral and of solutions of stochastic equations with Wick-type nonlinearities. In this paper we introduce and study bounded and unbounded operators of stochastic differentiation in the Levy white noise analysis. More exactly, we consider these operators on spaces from parametrized regular rigging of the space of square integrable with respect to the measure of a Levy white noise functions, using the Lytvynov's generalization of the chaotic representation property. This gives a possibility to extend to the Levy white noise analysis and to deepen the corresponding results of the classical white noise analysis.

On extended stochastic integrals with respect to Lévy processes

Let $L$ be a Levy process on $[0,+\infty)$. In particular cases, when $L$ is a Wiener or Poisson process, any square integrable random variable can be decomposed in a series of repeated stochastic integrals from nonrandom functions with respect to $L$. This property of $L$, known as the chaotic representation property (CRP), plays a very important role in the stochastic analysis. Unfortunately, for a general Levy process the CRP does not hold. There are different generalizations of the CRP for Levy processes. In particular, under the Ito's approach one decomposes a Levy process $L$ in the sum of a Gaussian process and a stochastic integral with respect to a Poisson random measure, and then uses the CRP for both terms in order to obtain a generalized CRP for $L$. The Nualart-Schoutens's approach consists in decomposition of a square integrable random variable in a series of repeated stochastic integrals from nonrandom functions with respect to so-called orthogonalized centered power jump processes, these processes are constructed with using of a cadlag version of $L$. The Lytvynov's approach is based on orthogonalization of continuous monomials in the space of square integrable random variables. In this paper we construct the extended stochastic integral with respect to a Levy process and the Hida stochastic derivative in terms of the Lytvynov's generalization of the CRP; establish some properties of these operators; and, what is most important, show that the extended stochastic integrals, constructed with use of the above-mentioned generalizations of the CRP, coincide.