Criteria for classifying general Markov chains

1976 ◽  
Vol 8 (04) ◽  
pp. 737-771 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
General State ◽  
Classical Work ◽  
The Status ◽  
General State Space

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Criteria for classifying general Markov chains

1976 ◽  
Vol 8 (4) ◽  
pp. 737-771 ◽  
Author(s):  
R. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
General State ◽  
Classical Work ◽  
The Status ◽  
General State Space

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b).

Quasi-stationary distributions for Markov chains on a general state space

1974 ◽  
Vol 11 (4) ◽  
pp. 726-741 ◽  
Author(s):  
Richard. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Invariant Measure ◽  
Markov Chains ◽  
State Space ◽  
Finiteness Condition ◽  
General State ◽  
Stationary Distributions ◽  
General State Space ◽  
Countably Infinite ◽  
Stationary Behaviour

The quasi-stationary behaviour of a Markov chain which is φ-irreducible when restricted to a subspace of a general state space is investigated. It is shown that previous work on the case where the subspace is finite or countably infinite can be extended to general chains, and the existence of certain quasi-stationary limits as honest distributions is equivalent to the restricted chain being R-positive with the unique R-invariant measure satisfying a certain finiteness condition.

Quasi-stationary distributions for Markov chains on a general state space

1974 ◽  
Vol 11 (04) ◽  
pp. 726-741 ◽  
Author(s):  
Richard. L. Tweedie
Keyword(s):  
Markov Chain ◽  
Invariant Measure ◽  
Markov Chains ◽  
State Space ◽  
Finiteness Condition ◽  
General State ◽  
Stationary Distributions ◽  
General State Space ◽  
Countably Infinite ◽  
Stationary Behaviour

The quasi-stationary behaviour of a Markov chain which is φ-irreducible when restricted to a subspace of a general state space is investigated. It is shown that previous work on the case where the subspace is finite or countably infinite can be extended to general chains, and the existence of certain quasi-stationary limits as honest distributions is equivalent to the restricted chain being R-positive with the unique R-invariant measure satisfying a certain finiteness condition.

PIGGYBACKING THRESHOLD PROCESSES WITH A FINITE STATE MARKOV CHAIN

2009 ◽  
Vol 09 (02) ◽  
pp. 187-204
Author(s):  
THOMAS R. BOUCHER ◽  
DAREN B. H. CLINE
Keyword(s):  
Time Series ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
General State ◽  
Stationary Distributions ◽  
Autoregressive Time Series ◽  
Finite State ◽  
Nonlinear Autoregressive

The state-space representations of certain nonlinear autoregressive time series are general state Markov chains. The transitions of a general state Markov chain among regions in its state-space can be modeled with the transitions among states of a finite state Markov chain. Stability of the time series is then informed by the stationary distributions of the finite state Markov chain. This approach generalizes some previous results.

Orthogonal measures and absorbing sets for Markov chains

1997 ◽  
Vol 121 (1) ◽  
pp. 101-113 ◽  
Author(s):  
PEI-DE CHEN ◽  
R. L. TWEEDIE
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Finite Measure ◽  
Countable Union ◽  
General State ◽  
Positive Mass ◽  
Absorbing Sets ◽  
General State Space ◽  
Orthogonal Measures

For a general state space Markov chain on a space (X, [Bscr ](X)), the existence of a Doeblin decomposition, implying the state space can be written as a countable union of absorbing ‘recurrent’ sets and a transient set, is known to be a consequence of several different conditions all implying in some way that there is not an uncountable collection of absorbing sets. These include([Mscr ]) there exists a finite measure which gives positive mass to each absorbing subset of X;([Gscr ]) there exists no uncountable collection of points (xα) such that the measures Kθ(xα, ·)[colone ](1−θ)ΣPn(xα, ·)θn are mutually singular;([Cscr ]) there is no uncountable disjoint class of absorbing subsets of X.We prove that if [Bscr ](X) is countably generated and separated (distinct elements in X can be separated by disjoint measurable sets), then these conditions are equivalent. Other results on the structure of absorbing sets are also developed.

scholarly journals Foster-type criteria for Markov chains on general spaces

2006 ◽  
Vol 43 (4) ◽  
pp. 1194-1200 ◽  
Author(s):  
Brian H. Fralix
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Open Problem ◽  
Stopping Times ◽  
Geometric Ergodicity ◽  
General State ◽  
Harris Recurrence ◽  
General State Space ◽  
Drift Conditions

This paper establishes new Foster-type criteria for a Markov chain on a general state space to be Harris recurrent, positive Harris recurrent, or geometrically ergodic. The criteria are based on drift conditions involving stopping times rather than deterministic steps. Meyn and Tweedie (1994) developed similar criteria involving random-sized steps, independent of the Markov chain under study. They also posed an open problem of finding criteria involving stopping times. Our results essentially solve that problem. We also show that the assumption of ψ-irreducibility is not needed when stating our drift conditions for positive Harris recurrence or geometric ergodicity.

scholarly journals Foster-type criteria for Markov chains on general spaces

2006 ◽  
Vol 43 (04) ◽  
pp. 1194-1200 ◽  
Author(s):  
Brian H. Fralix
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Open Problem ◽  
Stopping Times ◽  
Geometric Ergodicity ◽  
General State ◽  
Harris Recurrence ◽  
General State Space ◽  
Drift Conditions

This paper establishes new Foster-type criteria for a Markov chain on a general state space to be Harris recurrent, positive Harris recurrent, or geometrically ergodic. The criteria are based on drift conditions involving stopping times rather than deterministic steps. Meyn and Tweedie (1994) developed similar criteria involving random-sized steps, independent of the Markov chain under study. They also posed an open problem of finding criteria involving stopping times. Our results essentially solve that problem. We also show that the assumption of ψ-irreducibility is not needed when stating our drift conditions for positive Harris recurrence or geometric ergodicity.

Reversibility of Markov chains with applications to storage models

1985 ◽  
Vol 22 (01) ◽  
pp. 123-137 ◽  
Author(s):  
Hideo Ōsawa
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Discrete Time ◽  
Risk Model ◽  
Sufficient Conditions ◽  
Single Server ◽  
General State ◽  
Insurance Risk ◽  
General State Space ◽  
Necessary And Sufficient

This paper studies the reversibility conditions of stationary Markov chains (discrete-time Markov processes) with general state space. In particular, we investigate the Markov chains having atomic points in the state space. Such processes are often seen in storage models, for example waiting time in a queue, insurance risk reserve, dam content and so on. The necessary and sufficient conditions for reversibility of these processes are obtained. Further, we apply these conditions to some storage models and present some interesting results for single-server queues and a finite insurance risk model.

A New Algorithm for Computing the Ergodic Probability Vector for Large Markov Chains

1990 ◽  
Vol 4 (1) ◽  
pp. 89-116 ◽  
Author(s):  
Ushlo Sumita ◽  
Maria Rieders
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
The State ◽  
Special Structure ◽  
Probability Vector ◽  
Rank Reduction ◽  
Replacement Process ◽  
Novel Algorithm

A novel algorithm is developed which computes the ergodic probability vector for large Markov chains. Decomposing the state space into lumps, the algorithm generates a replacement process on each lump, where any exit from a lump is instantaneously replaced at some state in that lump. The replacement distributions are constructed recursively in such a way that, in the limit, the ergodic probability vector for a replacement process on one lump will be proportional to the ergodic probability vector of the original Markov chain restricted to that lump. Inverse matrices computed in the algorithm are of size (M – 1), where M is the number of lumps, thereby providing a substantial rank reduction. When a special structure is present, the procedure for generating the replacement distributions can be simplified. The relevance of the new algorithm to the aggregation-disaggregation algorithm of Takahashi [29] is also discussed.

Sign in / Sign up

Export Citation Format

Share Document