Stochastic optimal control with random coefficients and associated stochastic Hamilton–Jacobi–Bellman equations

We consider the optimal control problem for stochastic differential equations (SDEs) with random coefficients under the recursive-type objective functional captured by the backward SDE (BSDE). Due to the random coefficients, the associated Hamilton–Jacobi–Bellman (HJB) equation is a class of second-order stochastic PDEs (SPDEs) driven by Brownian motion, which we call the stochastic HJB (SHJB) equation. In addition, as we adopt the recursive-type objective functional, the drift term of the SHJB equation depends on the second component of its solution. These two generalizations cause several technical intricacies, which do not appear in the existing literature. We prove the dynamic programming principle (DPP) for the value function, for which unlike the existing literature we have to use the backward semigroup associated with the recursive-type objective functional. By the DPP, we are able to show the continuity of the value function. Using the Itô–Kunita’s formula, we prove the verification theorem, which constitutes a sufficient condition for optimality and characterizes the value function, provided that the smooth (classical) solution of the SHJB equation exists. In general, the smooth solution of the SHJB equation may not exist. Hence, we study the existence and uniqueness of the solution to the SHJB equation under two different weak solution concepts. First, we show, under appropriate assumptions, the existence and uniqueness of the weak solution via the Sobolev space technique, which requires converting the SHJB equation to a class of backward stochastic evolution equations. The second result is obtained under the notion of viscosity solutions, which is an extension of the classical one to the case for SPDEs. Using the DPP and the estimates of BSDEs, we prove that the value function is the viscosity solution to the SHJB equation. For applications, we consider the linear-quadratic problem, the utility maximization problem, and the European option pricing problem. Specifically, different from the existing literature, each problem is formulated by the generalized recursive-type objective functional and is subject to random coefficients. By applying the theoretical results of this paper, we obtain the explicit optimal solution for each problem in terms of the solution of the corresponding SHJB equation.

Equilibrium reinsurance-investment strategy with a common shock under two kinds of premium principles

This paper investigates the optimal mean-variance reinsurance-investment problem for an insurer with a common shock dependence under two kinds of popular premium principles: the variance premium principle and the expected value premium principle. We formulate the optimization problem within a game theoretic framework and derive the closed-form expressions of the equilibrium reinsurance-investment strategy and equilibrium value function under the two different premium principles by solving the extended Hamilton-Jacobi-Bellman system of equations. We find that under the variance premium principle, the proportional reinsurance is the optimal reinsurance strategy for the optimal reinsurance-investment problem with a common shock, while under the expected value premium principle, the excess-of-loss reinsurance is the optimal reinsurance strategy. In addition, we illustrate the equilibrium reinsurance-investment strategy by numerical examples and discuss the impacts of model parameters on the equilibrium strategy.

Optimal Investment and Consumption for Multidimensional Spread Financial Markets with Logarithmic Utility

We consider a spread financial market defined by the multidimensional Ornstein–Uhlenbeck (OU) process. We study the optimal consumption/investment problem for logarithmic utility functions using a stochastic dynamical programming method. We show a special verification theorem for this case. We find the solution to the Hamilton–Jacobi–Bellman (HJB) equation in explicit form and as a consequence we construct optimal financial strategies. Moreover, we study the constructed strategies with numerical simulations.

Deep Learning for Constrained Utility Maximisation

This paper proposes two algorithms for solving stochastic control problems with deep learning, with a focus on the utility maximisation problem. The first algorithm solves Markovian problems via the Hamilton Jacobi Bellman (HJB) equation. We solve this highly nonlinear partial differential equation (PDE) with a second order backward stochastic differential equation (2BSDE) formulation. The convex structure of the problem allows us to describe a dual problem that can either verify the original primal approach or bypass some of the complexity. The second algorithm utilises the full power of the duality method to solve non-Markovian problems, which are often beyond the scope of stochastic control solvers in the existing literature. We solve an adjoint BSDE that satisfies the dual optimality conditions. We apply these algorithms to problems with power, log and non-HARA utilities in the Black-Scholes, the Heston stochastic volatility, and path dependent volatility models. Numerical experiments show highly accurate results with low computational cost, supporting our proposed algorithms.

Continuity of the value function for deterministic optimal impulse control with terminal state constraint

Deterministic optimal impulse control problem with terminal state constraint is considered. Due to the appearance of the terminal state constraint, the value function might be discontinuous in general. The main contribution of this paper is the introduction of an intrinsic condition under which the value function is proved to be continuous. Then by a Bellman dynamic programming principle, the corresponding Hamilton-Jacobi-Bellman type quasi-variational inequality (QVI, for short) is derived. The value function is proved to be a viscosity solution to such a QVI. The issue of whether the value function is characterized as the unique viscosity solution to this QVI is carefully addressed and the answer is left open challengingly.

Critic-observer-based decentralized force/position approximate optimal control for modular and reconfigurable manipulators with uncertain environmental constraints

A critic-observer decentralized force/position approximate optimal control method is presented to address the joint trajectory and contacted force tracking problem of modular and reconfigurable manipulators (MRMs) with uncertain environmental constraints. The dynamic model of the MRM systems is formulated as an integration of joint subsystems via extensive state observer (ESO) associated with the effect of interconnected dynamic coupling (IDC). A radial basis function neural network (RBF-NN) is developed to deal with the IDC effects among the independent joint subsystems. Based on adaptive dynamic programming (ADP) approach and policy iteration (PI) algorithm, the Hamilton–Jacobi–Bellman (HJB) equation is approximately solved by establishing critic NN structure and then the approximated optimal control policy can be derived. The closed-loop manipulator system is proved to be asymptotic stable by using the Lyapunov theory. Finally, simulation results are provided to demonstrate the effectiveness and advantages of the proposed control method.

Mathematical and numerical analyses of a stochastic impulse control model with imperfect interventions

A stochastic impulse control problem with imperfect controllability of interventions is formulated with an emphasis on applications to ecological and environmental management problems. The imperfectness comes from uncertainties with respect to the magnitude of interventions. Our model is based on a dynamic programming formalism to impulsively control a 1-D diffusion process of a geometric Brownian type. The imperfectness leads to a non-local operator different from the many conventional ones, and evokes a slightly different optimal intervention policy. We give viscosity characterizations of the Hamilton–Jacobi–Bellman Quasi-Variational Inequality (HJBQVI) governing the value function focusing on its numerical computation. Uniqueness and verification results of the HJBQVI are presented and a candidate exact solution is constructed. The HJBQVI is solved with the two different numerical methods, an ordinary differential equation (ODE) based method and a finite difference scheme, demonstrating their consistency. Furthermore, the resulting controlled dynamics are extensively analyzed focusing on a bird population management case from a statistical standpoint.

Evolution of heterogeneity under constant and variable environments

Various definitions of fitness are essentially based on the number of descendants of an allele or a phenotype after a sufficiently long time. However, these different definitions do not explicate the continuous evolution of life histories. Herein, we focus on the eigenfunction of an age-structured population model as fitness. The function generates an equation, called the Hamilton–Jacobi–Bellman equation, that achieves adaptive control of life history in terms of both the presence and absence of the density effect. Further, we introduce a perturbation method that applies the solution of this equation to the long-term logarithmic growth rate of a stochastic structured population model. We adopt this method to realize the adaptive control of heterogeneity for an optimal foraging problem in a variable environment as the analyzable example. The result indicates that the eigenfunction is involved in adaptive strategies under all the environments listed herein. Thus, we aim to systematize adaptive life histories in the presence of density effects and variable environments using the proposed objective function as a universal fitness candidate.