Mirror descent algorithm on the indefinite control horizon

We consider the problem of optimal control in a random environment in a minimax setting as applied to data processing. It is assumed that the random environment provides two methods of data processing, the effectiveness of which is not known in advance. The goal of the control in this case is to find the optimal strategy for the application of processing methods and to minimize losses. To solve this problem, the mirror descent algorithm is used, including its modifications for batch processing. The use of algorithms for batch processing allows us to get a significant gain in speed due to the parallel processing of batches. In the classical statement, the search for the optimal strategy is considered on a fixed control horizon but this article considers an indefinite control horizon. With an indefinite horizon, the control algorithm cannot use information about the value of the horizon when searching for an optimal strategy. Using numerical modeling, the operation of the mirror descent algorithm and its modifications on an indefinite control horizon is studied and obtained results are presented.

Two-armed bandit problem and batch version of the mirror descent algorithm

We consider the minimax setup for the two-armed bandit problem as applied to data processing if there are two alternative processing methods with different a priori unknown efficiencies. One should determine the most efficient method and provide its predominant application. To this end, we use the mirror descent algorithm (MDA). It is well-known that corresponding minimax risk has the order of $N^{1/2$ with $N$ being the number of processed data and this bound is unimprovable in order. We propose a batch version of the MDA which allows processing data by packets that is especially important if parallel data processing can be provided. In this case, the processing time is determined by the number of batches rather than by the total number of data. Unexpectedly, it turned out that the batch version behaves unlike the ordinary one even if the number of packets is large. Moreover, the batch version provides significantly smaller value of the minimax risk, i.e., it considerably improves a control performance. We explain this result by considering another batch modification of the MDA which behavior is close to behavior of the ordinary version and minimax risk is close as well. Our estimates use invariant descriptions of the algorithms based on Gaussian approximations of incomes in batches of data in the domain of ``close'' distributions and are obtained by Monte-Carlo simulations.

Algorithms for Convex Optimization

In the last few years, Algorithms for Convex Optimization have revolutionized algorithm design, both for discrete and continuous optimization problems. For problems like maximum flow, maximum matching, and submodular function minimization, the fastest algorithms involve essential methods such as gradient descent, mirror descent, interior point methods, and ellipsoid methods. The goal of this self-contained book is to enable researchers and professionals in computer science, data science, and machine learning to gain an in-depth understanding of these algorithms. The text emphasizes how to derive key algorithms for convex optimization from first principles and how to establish precise running time bounds. This modern text explains the success of these algorithms in problems of discrete optimization, as well as how these methods have significantly pushed the state of the art of convex optimization itself.