scholarly journals Efficient Method to Approximately Solve Retrial Systems with Impatience

2012 ◽  
Vol 2012 ◽  
pp. 1-18
Author(s):  
Jose Manuel Gimenez-Guzman ◽  
M. Jose Domenech-Benlloch ◽  
Vicent Pla ◽  
Jorge Martinez-Bauset ◽  
Vicente Casares-Giner
Keyword(s):  
State Space ◽  
Decision Process ◽  
Analytic Solution ◽  
Computational Cost ◽  
Continuous Time Markov Chain ◽  
Kolmogorov Equations ◽  
Novel Technique ◽  
Polynomial Extrapolation ◽  
Markov Decision ◽  
Infinite State

We present a novel technique to solve multiserver retrial systems with impatience. Unfortunately these systems do not present an exact analytic solution, so it is mandatory to resort to approximate techniques. This novel technique does not rely on the numerical solution of the steady-state Kolmogorov equations of the Continuous Time Markov Chain as it is common for this kind of systems but it considers the system in its Markov Decision Process setting. This technique, known as value extrapolation, truncates the infinite state space using a polynomial extrapolation method to approach the states outside the truncated state space. A numerical evaluation is carried out to evaluate this technique and to compare its performance with previous techniques. The obtained results show that value extrapolation greatly outperforms the previous approaches appeared in the literature not only in terms of accuracy but also in terms of computational cost.

scholarly journals Numerical approximation for the solution of linear sixth order boundary value problems by cubic B-spline

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
A. Khalid ◽  
M. N. Naeem ◽  
P. Agarwal ◽  
A. Ghaffar ◽  
Z. Ullah ◽  
...  
Keyword(s):  
Numerical Approximation ◽  
Analytic Solution ◽  
Linear Equations ◽  
Computational Cost ◽  
Spline Method ◽  
System Of Linear Equations ◽  
Approximation Solution ◽  
B Spline ◽  
Novel Technique ◽  
Derivatives Of

AbstractIn the current paper, authors proposed a computational model based on the cubic B-spline method to solve linear 6th order BVPs arising in astrophysics. The prescribed method transforms the boundary problem to a system of linear equations. The algorithm we are going to develop in this paper is not only simply the approximation solution of the 6th order BVPs using cubic B-spline, but it also describes the estimated derivatives of 1st order to 6th order of the analytic solution at the same time. This novel technique has lesser computational cost than numerous other techniques and is second order convergent. To show the efficiency of the proposed method, four numerical examples have been tested. The results are described using error tables and graphs and are compared with the results existing in the literature.

scholarly journals Generalized Truncated Methods for an Efficient Solution of Retrial Systems

2008 ◽  
Vol 2008 ◽  
pp. 1-15 ◽  
Author(s):  
Ma Jose Domenech-Benlloch ◽  
Jose Manuel Gimenez-Guzman ◽  
Vicent Pla ◽  
Jorge Martinez-Bauset ◽  
Vicente Casares-Giner
Keyword(s):  
Closed Form ◽  
State Space ◽  
Efficient Solution ◽  
Analytic Solution ◽  
Computational Cost ◽  
Retrial Queues ◽  
Performance Parameters ◽  
Approximate Methods ◽  
Closed Form Solutions ◽  
Wide Range

We are concerned with the analytic solution of multiserver retrial queues including the impatience phenomenon. As there are not closed-form solutions to these systems, approximate methods are required. We propose two different generalized truncated methods to effectively solve this type of systems. The methods proposed are based on the homogenization of the state space beyond a given number of users in the retrial orbit. We compare the proposed methods with the most well-known methods appeared in the literature in a wide range of scenarios. We conclude that the proposed methods generally outperform previous proposals in terms of accuracy for the most common performance parameters used in retrial systems with a moderated growth in the computational cost.

scholarly journals Solving Continual Combinatorial Selection via Deep Reinforcement Learning

2019 ◽  
Author(s):  
Hyungseok Song ◽  
Hyeryung Jang ◽  
Hai H. Tran ◽  
Se-eun Yoon ◽  
Kyunghwan Son ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Markov Decision Process ◽  
Decision Process ◽  
Joint Action ◽  
Expressive Power ◽  
State Space Explosion ◽  
Exponential Increase ◽  
Markov Decision ◽  
Action Spaces

We consider the Markov Decision Process (MDP) of selecting a subset of items at each step, termed the Select-MDP (S-MDP). The large state and action spaces of S-MDPs make them intractable to solve with typical reinforcement learning (RL) algorithms especially when the number of items is huge. In this paper, we present a deep RL algorithm to solve this issue by adopting the following key ideas. First, we convert the original S-MDP into an Iterative Select-MDP (IS-MDP), which is equivalent to the S-MDP in terms of optimal actions. IS-MDP decomposes a joint action of selecting K items simultaneously into K iterative selections resulting in the decrease of actions at the expense of an exponential increase of states. Second, we overcome this state space explosion by exploiting a special symmetry in IS-MDPs with novel weight shared Q-networks, which provably maintain sufficient expressive power. Various experiments demonstrate that our approach works well even when the item space is large and that it scales to environments with item spaces different from those used in training.

scholarly journals Embedding a state space model into a Markov decision process

2010 ◽  
Vol 190 (1) ◽  
pp. 289-309 ◽  
Author(s):  
Lars Relund Nielsen ◽  
Erik Jørgensen ◽  
Søren Højsgaard
Keyword(s):  
State Space ◽  
Markov Decision Process ◽  
Decision Process ◽  
State Space Model ◽  
Space Model ◽  
Markov Decision

Weak convergence of conditioned processes on a countable state space

1995 ◽  
Vol 32 (4) ◽  
pp. 902-916 ◽  
Author(s):  
S. D. Jacka ◽  
G. O. Roberts
Keyword(s):  
Weak Convergence ◽  
State Space ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Original Process ◽  
Countable State ◽  
Positive Time ◽  
Countably Infinite ◽  
Infinite State

We consider the problem of conditioning a continuous-time Markov chain (on a countably infinite state space) not to hit an absorbing barrier before time T; and the weak convergence of this conditional process as T → ∞. We prove a characterization of convergence in terms of the distribution of the process at some arbitrary positive time, t, introduce a decay parameter for the time to absorption, give an example where weak convergence fails, and give sufficient conditions for weak convergence in terms of the existence of a quasi-stationary limit, and a recurrence property of the original process.

scholarly journals A Dynamic Strategy for Home Pick-Up Service with Uncertain Customer Requests and Its Implementation

2019 ◽  
Vol 11 (7) ◽  
pp. 2060
Author(s):  
Yu Wu ◽  
Bo Zeng ◽  
Siming Huang
Keyword(s):  
Dynamic Programming ◽  
State Space ◽  
Markov Decision Process ◽  
Decision Process ◽  
Solution Method ◽  
Computing Complexity ◽  
Home Service ◽  
Dynamic Strategy ◽  
Markov Decision ◽  
Capacitated Vehicle

In this paper, a home service problem is studied, where a capacitated vehicle collects customers’ parcels in one pick-up tour. We consider a situation where customers, who have scheduled their services in advance, may call to cancel their appointments, and customers, who do not have appointments, also need to be visited if they request for services as long as the capacity is allowed. To handle those changes that occurred over the tour, a dynamic strategy will be needed to guide the vehicle to visit customers in an efficient way. Aimed at minimizing the vehicle’s total expected travel distance, we model this problem as a multi-dimensional Markov Decision Process (MDP) with finite exponential scale state space. We exactly solve this MDP via dynamic programming, where the computing complexity is exponential. In order to avoid complexity continually increasing, we aim to develop a fast looking-up method for one already-examined state’s record. Although generally this will result in a huge waste of memory, by exploiting critical structural properties of the state space, we obtain an O ( 1 ) looking-up method without any waste of memory. Computational experiments demonstrate the effectiveness of our model and the developed solution method. For larger instances, two well-performed heuristics are proposed.

Sign in / Sign up

Export Citation Format

Share Document