Recurrent Multinomial Models for Categorical Sequences

This paper presents a model of recurrent multinomial sequences. Though there exists a quite considerable literature on modeling autocorrelation in numerical data and sequences of categorical outcomes, there is currently no systematic method of modeling patterns of recurrence in categorical sequences. This paper develops a means of discovering recurrent patterns by employing a more restrictive Markov assumption. The resulting model, which I call the recurrent multinomial model, provides a parsimonious representation of recurrent sequences, enabling the investigation of recurrences on longer time scales than existing models. The utility of recurrent multinomial models is demonstrated by applying them to the case of conversational turn-taking in meetings of the Federal Open Market Committee (FOMC). Analyses are effectively able to discover norms around turn-reclaiming, participation, and suppression and to evaluate how these norms vary throughout the course of the meeting.

A hybrid landmark Aalen-Johansen estimator for transition probabilities in partially non-Markov multi-state models

Multi-state models are increasingly being used to model complex epidemiological and clinical outcomes over time. It is common to assume that the models are Markov, but the assumption can often be unrealistic. The Markov assumption is seldomly checked and violations can lead to biased estimation of many parameters of interest. This is a well known problem for the standard Aalen-Johansen estimator of transition probabilities and several alternative estimators, not relying on the Markov assumption, have been suggested. A particularly simple approach known as landmarking have resulted in the Landmark-Aalen-Johansen estimator. Since landmarking is a stratification method a disadvantage of landmarking is data reduction, leading to a loss of power. This is problematic for “less traveled” transitions, and undesirable when such transitions indeed exhibit Markov behaviour. Introducing the concept of partially non-Markov multi-state models, we suggest a hybrid landmark Aalen-Johansen estimator for transition probabilities. We also show how non-Markov transitions can be identified using a testing procedure. The proposed estimator is a compromise between regular Aalen-Johansen and landmark estimation, using transition specific landmarking, and can drastically improve statistical power. We show that the proposed estimator is consistent, but that the traditional variance estimator can underestimate the variance of both the hybrid and landmark estimator. Bootstrapping is therefore recommended. The methods are compared in a simulation study and in a real data application using registry data to model individual transitions for a birth cohort of 184 951 Norwegian men between states of sick leave, disability, education, work and unemployment.

Inferring Time-delayed Causal Relations in POMDPs from the Principle of Independence of Cause and Mechanism

This paper introduces an algorithm for discovering implicit and delayed causal relations between events observed by a robot at regular or arbitrary times, with the objective of improving data-efficiency and interpretability of model-based reinforcement learning (RL) techniques. The proposed algorithm initially predicts observations with the Markov assumption, and incrementally introduces new hidden variables to explain and reduce the stochasticity of the observations. The hidden variables are memory units that keep track of pertinent past events. Such events are systematically identified by their information gains. A test of independence between inputs and mechanisms is performed to identify cases when there is a causal link between events and those when the information gain is due to confounding variables. The learned transition and reward models are then used in a Monte Carlo tree search for planning. Experiments on simulated and real robotic tasks, and the challenging 3D game Doom show that this method significantly improves over current RL techniques.

Estimation of covariate effects on net survivals in the relative survival progressive illness-death model

Recently, there has been a lot of development in relative survival field. In the absence of data on the cause of death, the research has tended to focus on the estimation of survival probability of a cancer (as a disease of interest). In many cancers, one nonfatal event that decreases the survival probability can occur. There are a few methods that assess the role of prognostic factors for multiple types of clinical events while dealing with uncertainty about the cause of death. However, these methods require proportional hazard or Markov assumptions. In practice, one or both of these assumptions might be violated. Violation of the proportional hazard assumption can lead to estimates that are biased, and difficult to interpret and violation of Markov assumption results in inconsistent estimators. In this work, we propose a semi-parametric approach to estimate the possibly time-varying regression coefficients in the likely non-Markov relative survival progressive illness-death model. The performance of the proposed estimator is investigated through simulations. We illustrate our approach using data from a study on rectal cancer resected for cure conducted in two French population-based digestive cancer registries.

A situation-modulated minimal change account for causal inferences about causal networks

Although causal Bayes networks are applicable to examining causal inferences about different static objects and about a changing object with different states, previous studies investigated the former, but not the latter. We propose a situation-modulated minimal change account for causal inferences. It predicts that dynamic situations are more likely to elicit minimal revisions on causal networks and adherence to the Markov assumption than static situations. Two experiments were conducted to investigate qualitative causal inferences about causal networks with binary and numerical variables, respectively. It was found that qualitative causal inferences were more likely to adhere to the Markov assumption in dynamic situations than in static situations. This finding supports the situation-modulated minimal change account rather than the other alternative accounts. We conclude that dynamic situations are more likely to elicit minimal revisions on causal networks and adherence to the Markov assumption than static situations. This conclusion is beyond the previous predominant view that causal inferences are apt to violate the Markov assumption.

Estimation of the Transition Probabilities in Multi-state Survival Data: New Developments and Practical Recommendations

Multi-state models can be successfully used for describing complicated event history data, for example, describing stages in the disease progression of a patient. In these models one important goal is the estimation of the transition probabilities since they allow for long term prediction of the process. Traditionally these quantities have been estimated by the Aalen-Johansen estimator which is consistent if the process is Markovian. Recently, estimators have been proposed that outperform the Aalen-Johansen estimators in non-Markov situations. This paper considers a new proposal for the estimation of the transition probabilities in a multi-state system that is not necessarily Markovian. The proposed product-limit nonparametric estimator is defined in the form of a counting process, counting the number of transitions between states and the risk sets for leaving each state with an inverse probability of censoring weighted form. Advantages and limitations of the different methods and some practical recommendations are presented. We also introduce a graphical local test for the Markov assumption. Several simulation studies were conducted under different data scenarios. The proposed methods are illustrated with a real data set on colon cancer.

Using Markov assumption with covariates to assess the Plasmodium falciparum malaria serological markers evolution

In this study, we develop Three Markov models which are continuous time-homogeneous Model, time piecewise constant intensities Markov model and semi-Markov model with Weibull distribution as the waiting time distribution to evaluate malaria serology evolution. We consider two-state model describing antibody reactivity defined by immunologists. We discuss in detail the application of these models to identify relationships between malaria control program and serological measurements of malaria transmission

Using Markov assumption with covariates to assess the Plasmodium falciparum malaria serological markers evolution

In this study, we develop Three Markov models which are continuous time-homogeneous Model, time piecewise constant intensities Markov model and semi-Markov model with Weibull distribution as the waiting time distribution to evaluate malaria serology evolution. We consider two-state model describing antibody reactivity defined by immunologists. We discuss in detail the application of these models to identify relationships between malaria control program and serological measurements of malaria transmission