Equivalence of two absorption problems with Markovian transitions and continuous or discrete time parameters

1959 ◽  
Vol 55 (2) ◽  
pp. 177-180 ◽  
Author(s):  
R. A. Sack
Keyword(s):  
Finite Number ◽  
Markov Processes ◽  
Discrete Time ◽  
Continuous Time ◽  
Time Parameter ◽  
Rate Of Change ◽  
Matrix Form ◽  
Constant Matrix ◽  
Time Parameters ◽  
Image Position

1. Introduction. Ledermann(1) has treated the problem of calculating the asymptotic probabilities that a system will be found in any one of a finite number N of possible states if transitions between these states occur as Markov processes with a continuous time parameter t. If we denote by pi(t) the probability that at time t the system is in the ith state and by aij ( ≥ 0) the constant probability per unit time for transitions from the jth to the ith state, the rate of change of pi is given bywhere the sum is to be taken over all j ≠ i. This set of equations can be written in matrix form aswhere P(t) is the vector with components pi(t) and the constant matrix A has elements

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

Proportional intensities and strong ergodicity for Markov processes

1983 ◽  
Vol 20 (01) ◽  
pp. 185-190 ◽  
Author(s):  
Mark Scott ◽  
Dean L. Isaacson
Keyword(s):  
Markov Process ◽  
Markov Processes ◽  
Discrete Time ◽  
Continuous Time ◽  
Homogeneous Markov Process ◽  
Strong Ergodicity ◽  
Transition Functions ◽  
The Matrix ◽  
Intensity Functions ◽  
Time Point

By assuming the proportionality of the intensity functions at each time point for a continuous-time non-homogeneous Markov process, strong ergodicity for the process is determined through strong ergodicity of a related discrete-time Markov process. For processes having proportional intensities, strong ergodicity implies having the limiting matrix L satisfy L · P(s, t) = L, where P(s, t) is the matrix of transition functions.

Homecomings of Markov processes

1973 ◽  
Vol 5 (01) ◽  
pp. 66-102 ◽  
Author(s):  
J. F. C. Kingman
Keyword(s):  
Markov Process ◽  
Markov Processes ◽  
Continuous Time ◽  
Random Time ◽  
Random Set ◽  
Theoretical Interest ◽  
Different Types ◽  
Treat All ◽  
Image Position

Ifx0is a particular state for a continuous-time Markov processX, the random time setis often of both practical and theoretical interest. Ignoring trivial or pathological cases, there are four different types of structure which this random set can display. To some extent, it is possible to treat all four cases in a unified way, but they raise different questions and require different modes of description. The distributions of various random quantities associated withcan be related to one another by simple and useful formulae.

Limit theorems for processes such as the Markov branching process

1975 ◽  
Vol 12 (02) ◽  
pp. 289-297
Author(s):  
Andrew D. Barbour
Keyword(s):  
Markov Process ◽  
Residence Time ◽  
Continuous Time ◽  
Limit Theorems ◽  
Branching Process ◽  
Continuous Mapping ◽  
Random Variables ◽  
Time Parameter ◽  
Image Position ◽  
Independent Identically Distributed

LetX(t) be a continuous time Markov process on the integers such that, ifσis a time at whichXmakes a jump,X(σ)– X(σ–) is distributed independently ofX(σ–), and has finite meanμand variance. Letq(j) denote the residence time parameter for the statej.Iftndenotes the time of thenth jump andXn≡X(tb), it is easy to deduce limit theorems forfrom those for sums of independent identically distributed random variables. In this paper, it is shown how, forμ> 0 and for suitableq(·), these theorems can be translated into limit theorems forX(t), by using the continuous mapping theorem.

Continuous-time allocation indices and their discrete-time approximation

1986 ◽  
Vol 18 (03) ◽  
pp. 724-746
Author(s):  
W. J. R. Eplett
Keyword(s):  
Markov Processes ◽  
Discrete Time ◽  
Time Allocation ◽  
Optimal Policy ◽  
Continuous Time ◽  
Bandit Problems ◽  
Time Approximation ◽  
Discrete Time Approximation ◽  
Strong Markov

The theory of allocation indices for defining the optimal policy in multi-armed bandit problems developed by Gittins is presented in the continuous-time case where the projects (or ‘arms’) are strong Markov processes. Complications peculiar to the continuous-time case are discussed. This motivates investigation of whether approximation of the continuous-time problems by discrete-time versions provides a valid technique with convergent allocation indices and optimal expected rewards. Conditions are presented under which the convergence holds.

On some waiting time problems

2000 ◽  
Vol 37 (03) ◽  
pp. 756-764 ◽  
Author(s):  
Valeri T. Stefanov
Keyword(s):  
Survival Analysis ◽  
Finite Number ◽  
Markov Processes ◽  
Waiting Time ◽  
Generating Functions ◽  
Continuous Time ◽  
Waiting Times ◽  
Phase Type ◽  
Phase Type Distributions ◽  
Semi Markov Processes

A unifying technology is introduced for finding explicit closed form expressions for joint moment generating functions of various random quantities associated with some waiting time problems. Sooner and later waiting times are covered for general discrete- and continuous-time models. The models are either Markov chains or semi-Markov processes with a finite number of states. Waiting times associated with generalized phase-type distributions, that are of interest in survival analysis and other areas, are also covered.

Sign in / Sign up

Export Citation Format

Share Document