Observable Operator Models for Discrete Stochastic Time Series

2000 ◽  
Vol 12 (6) ◽  
pp. 1371-1398 ◽  
Author(s):  
Herbert Jaeger
Keyword(s):  
Hidden Markov Models ◽  
Stochastic Systems ◽  
Markov Models ◽  
Learning Algorithm ◽  
Hidden Markov ◽  
Training Data ◽  
Constructive Learning ◽  
Proper Subclass ◽  
Stochastic Time Series ◽  
Dependent Processes

A widely used class of models for stochastic systems is hidden Markov models. Systems that can be modeled by hidden Markov models are a proper subclass of linearly dependent processes, a class of stochastic systems known from mathematical investigations carried out over the past four decades. This article provides a novel, simple characterization of linearly dependent processes, called observable operator models. The mathematical properties of observable operator models lead to a constructive learning algorithm for the identification of linearly dependent processes. The core of the algorithm has a time complexity of O (N + nm3), where N is the size of training data, n is the number of distinguishable outcomes of observations, and m is model state-space dimension.

scholarly journals Smooth On-Line Learning Algorithms for Hidden Markov Models

1994 ◽  
Vol 6 (2) ◽  
pp. 307-318 ◽  
Author(s):  
Pierre Baldi ◽  
Yves Chauvin
Keyword(s):  
Hidden Markov Models ◽  
Expectation Maximization ◽  
Markov Models ◽  
Learning Algorithm ◽  
Hidden Markov ◽  
Batch Mode ◽  
On Line ◽  
Maximization Algorithms ◽  
On Line Learning ◽  
Entropy Functions

A simple learning algorithm for Hidden Markov Models (HMMs) is presented together with a number of variations. Unlike other classical algorithms such as the Baum-Welch algorithm, the algorithms described are smooth and can be used on-line (after each example presentation) or in batch mode, with or without the usual Viterbi most likely path approximation. The algorithms have simple expressions that result from using a normalized-exponential representation for the HMM parameters. All the algorithms presented are proved to be exact or approximate gradient optimization algorithms with respect to likelihood, log-likelihood, or cross-entropy functions, and as such are usually convergent. These algorithms can also be casted in the more general EM (Expectation-Maximization) framework where they can be viewed as exact or approximate GEM (Generalized Expectation-Maximization) algorithms. The mathematical properties of the algorithms are derived in the appendix.

Semi-supervised learning of Hidden Markov Models for biological sequence analysis

2018 ◽  
Vol 35 (13) ◽  
pp. 2208-2215 ◽  
Author(s):  
Ioannis A Tamposis ◽  
Konstantinos D Tsirigos ◽  
Margarita C Theodoropoulou ◽  
Panagiota I Kontou ◽  
Pantelis G Bagos
Keyword(s):  
Sequence Analysis ◽  
Supervised Learning ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Transmembrane Protein ◽  
Training Data ◽  
Supplementary Information ◽  
Training Procedure ◽  
Partially Labeled Data

Abstract Motivation Hidden Markov Models (HMMs) are probabilistic models widely used in applications in computational sequence analysis. HMMs are basically unsupervised models. However, in the most important applications, they are trained in a supervised manner. Training examples accompanied by labels corresponding to different classes are given as input and the set of parameters that maximize the joint probability of sequences and labels is estimated. A main problem with this approach is that, in the majority of the cases, labels are hard to find and thus the amount of training data is limited. On the other hand, there are plenty of unclassified (unlabeled) sequences deposited in the public databases that could potentially contribute to the training procedure. This approach is called semi-supervised learning and could be very helpful in many applications. Results We propose here, a method for semi-supervised learning of HMMs that can incorporate labeled, unlabeled and partially labeled data in a straightforward manner. The algorithm is based on a variant of the Expectation-Maximization (EM) algorithm, where the missing labels of the unlabeled or partially labeled data are considered as the missing data. We apply the algorithm to several biological problems, namely, for the prediction of transmembrane protein topology for alpha-helical and beta-barrel membrane proteins and for the prediction of archaeal signal peptides. The results are very promising, since the algorithms presented here can significantly improve the prediction performance of even the top-scoring classifiers. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals A New Data Fusion Method for Hybrid MMC/RNA Learning : Application to Automatic Speech Recognition

2005 ◽  
Vol Volume 3, Special Issue... ◽  
Author(s):  
Lilia Lazli ◽  
Mohamed Tayeb Laskri
Keyword(s):  
Neural Networks ◽  
Speech Recognition ◽  
Data Fusion ◽  
Hidden Markov Models ◽  
Markov Models ◽  
Hidden Markov ◽  
Recognition System ◽  
Training Data ◽  
Nous Rapportons ◽  
The One

International audience It is well known that traditional Hidden Markov Models (HMM) systems lead to a considerable improvement when more training data or more parameters are used. However, using more data with hybrid Hidden Markov Models and Artificial Neural Networks (HMM/ANN) models results in increased training times without improvements in performance. We developed in this work a new method based on automatically separating data into several sets and training several neural networks of Multi-Layer Perceptrons (MLP) type on each set. During the recognition phase, models are combined using several criteria (based on data fusion techniques) to provide the recognized word. We showed in this paper that this method significantly improved the recognition accuracy. This method was applied in an Arabic speech recognition system. This last is based on the one hand, on a fuzzy clustering (application of the fuzzy c-means algorithm) and of another share, on a segmentation at base of the genetic algorithms. De nombreuses expériences ont déjà montré qu'une forte amélioration du taux de reconnaissance des systèmes MMC (Modèles de Markov Cachés) traditionnels est observée lorsque plus de données d'apprentissage sont utilisées. En revanche, l'augmentation du nombre de données d'apprentissage pour les modèles hybrides MMC/RNA (Modèles de Markov cachés/Réseaux de Neurones Artificiels) s'accompagne d'une forte augmentation du temps nécessaire à l'apprentissage des modèles, mais pas ou peu des performances du système. Pour pallier cette limitation, nous rapportons dans ce papier les résultats obtenus avec une nouvelle méthode d'apprentissage basée sur la fusion de données. Cette méthode a été appliquée dans un système de reconnaissance de la parole arabe. Ce dernier est basé d'une part, sur une segmentation floue (application de l'algorithme c-moyennes floues) et d'une autre part, sur une segmentation à base des algorithmes génétiques.

Sign in / Sign up

Export Citation Format

Share Document