Singularly perturbed Markov chains. II. Applications to controlled dynamic systems and Markov decision processes

2002 ◽  
Author(s):  
Q. Zhang ◽  
G. Yin
Keyword(s):  
Markov Chains ◽  
Markov Decision Processes ◽  
Dynamic Systems ◽  
Decision Processes ◽  
Singularly Perturbed ◽  
Markov Decision

scholarly journals EQUIVALENCE OF LABELED MARKOV CHAINS

2008 ◽  
Vol 19 (03) ◽  
pp. 549-563 ◽  
Author(s):  
LAURENT DOYEN ◽  
THOMAS A. HENZINGER ◽  
JEAN-FRANÇOIS RASKIN
Keyword(s):  
Markov Chains ◽  
Markov Decision Processes ◽  
Polynomial Time ◽  
Finite Sequence ◽  
Equivalence Problem ◽  
Decision Processes ◽  
Probabilistic Automata ◽  
Alternative Algorithm ◽  
Markov Decision ◽  
Definition Of

We consider the equivalence problem for labeled Markov chains (LMCs), where each state is labeled with an observation. Two LMCs are equivalent if every finite sequence of observations has the same probability of occurrence in the two LMCs. We show that equivalence can be decided in polynomial time, using a reduction to the equivalence problem for probabilistic automata, which is known to be solvable in polynomial time. We provide an alternative algorithm to solve the equivalence problem, which is based on a new definition of bisimulation for probabilistic automata. We also extend the technique to decide the equivalence of weighted probabilistic automata. Then, we consider the equivalence problem for labeled Markov decision processes (LMDPs), which asks given two LMDPs whether for every scheduler (i.e. way of resolving the nondeterministic decisions) for each of the processes, there exists a scheduler for the other process such that the resulting LMCs are equivalent. The decidability of this problem remains open. We show that the schedulers can be restricted to be observation-based, but may require infinite memory.

scholarly journals Singularly perturbed linear programs and Markov decision processes

2016 ◽  
Vol 44 (3) ◽  
pp. 297-301
Author(s):  
Konstantin Avrachenkov ◽  
Jerzy A. Filar ◽  
Vladimir Gaitsgory ◽  
Andrew Stillman
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Singularly Perturbed ◽  
Linear Programs ◽  
Markov Decision
Sign in / Sign up

Export Citation Format

Share Document