Filtering method for linear and non-linear stochastic optimal control of partially observable systems II

In this paper we studied stochastic optimal control problem based on partially observable systems (SOCPP) with a control factor on the diffusion term. A SOCPP has state and observation processes. This kind of problem has also a minimum payoff function. The payoff function should be minimized according to the partially observable systems consist of the state and observation processes. In this regard, the filtering method is used to evaluat this kind of problem and express full consideration of it. Finally, presented estimation methods are used to simulate the solution of a partially observable system corresponding to the control factor of this problem. These methods are numerically used to solve linear and nonlinear cases.

Filtering method for linear and non-linear stochastic optimal control of partially observable systems

This paper studies two linear methods for linear and non-linear stochastic optimal control of partially observable problem (SOCPP). At first, it introduces the general form of a SOCPP and states it as a functional matrix. A SOCPP has a payoff function which should be minimized. It also has two dynamic processes: state and observation. In this study, it is presented a deterministic method to find the control factor which has named feedback control and stated a modified complete proof of control optimality in a general SOCPP. After finding the optimal control factor, it should be substituted in the state process to make the partially observable system. Next, it introduces a linear filtering method to solve the related partially observable system with complete details. Finally, it is presented a heuristic method in discrete form for estimating non-linear SOCPPs and it is stated some examples to evaluate the performance of introducing methods.

On-line parameter estimation for a failure-prone system subject to condition monitoring

In this paper, we study the on-line parameter estimation problem for a partially observable system subject to deterioration and random failure. The state of the system evolves according to a continuous-time homogeneous Markov process with a finite state space. The state of the system is hidden except for the failure state. When the system is operating, only the information obtained by condition monitoring, which is related to the working state of the system, is available. The condition monitoring observations are assumed to be in continuous range, so that no discretization is required. A recursive maximum likelihood (RML) algorithm is proposed for the on-line parameter estimation of the model. The new RML algorithm proposed in the paper is superior to other RML algorithms in the literature in that no projection is needed and no calculation of the gradient on the surface of the constraint manifolds is required. A numerical example is provided to illustrate the algorithm.

On-line parameter estimation for a failure-prone system subject to condition monitoring

In this paper, we study the on-line parameter estimation problem for a partially observable system subject to deterioration and random failure. The state of the system evolves according to a continuous-time homogeneous Markov process with a finite state space. The state of the system is hidden except for the failure state. When the system is operating, only the information obtained by condition monitoring, which is related to the working state of the system, is available. The condition monitoring observations are assumed to be in continuous range, so that no discretization is required. A recursive maximum likelihood (RML) algorithm is proposed for the on-line parameter estimation of the model. The new RML algorithm proposed in the paper is superior to other RML algorithms in the literature in that no projection is needed and no calculation of the gradient on the surface of the constraint manifolds is required. A numerical example is provided to illustrate the algorithm.