scholarly journals On Quantization Errors in Approximate and Sample Entropy

Entropy ◽  
2021 ◽  
Vol 24 (1) ◽  
pp. 73
Author(s):  
Dragana Bajić ◽  
Nina Japundžić-Žigon
Keyword(s):  
Time Series ◽  
Approximate Entropy ◽  
Large Error ◽  
Sample Entropy ◽  
Error Distribution ◽  
Series Length ◽  
Entropy Error ◽  
Coarse Quantization ◽  
True Values ◽  
Matching Probability

Approximate and sample entropies are acclaimed tools for quantifying the regularity and unpredictability of time series. This paper analyses the causes of their inconsistencies. It is shown that the major problem is a coarse quantization of matching probabilities, causing a large error between their estimated and true values. Error distribution is symmetric, so in sample entropy, where matching probabilities are directly summed, errors cancel each other. In approximate entropy, errors are accumulating, as sums involve logarithms of matching probabilities. Increasing the time series length increases the number of quantization levels, and errors in entropy disappear both in approximate and in sample entropies. The distribution of time series also affects the errors. If it is asymmetric, the matching probabilities are asymmetric as well, so the matching probability errors cease to be mutually canceled and cause a persistent entropy error. Despite the accepted opinion, the influence of self-matching is marginal as it just shifts the error distribution along the error axis by the matching probability quant. Artificial lengthening the time series by interpolation, on the other hand, induces large error as interpolated samples are statistically dependent and destroy the level of unpredictability that is inherent to the original signal.

scholarly journals Low Computational Cost for Sample Entropy

Entropy ◽  
2018 ◽  
Vol 20 (1) ◽  
pp. 61 ◽  
Author(s):  
George Manis ◽  
Md Aktaruzzaman ◽  
Roberto Sassi
Keyword(s):  
Time Series ◽  
Dimensional Space ◽  
A Priori ◽  
Computational Cost ◽  
Approximate Entropy ◽  
Sample Entropy ◽  
Series Length ◽  
Approximation Techniques ◽  
Computationally Intensive ◽  
Definition Of

Sample Entropy is the most popular definition of entropy and is widely used as a measure of the regularity/complexity of a time series. On the other hand, it is a computationally expensive method which may require a large amount of time when used in long series or with a large number of signals. The computationally intensive part is the similarity check between points in m dimensional space. In this paper, we propose new algorithms or extend already proposed ones, aiming to compute Sample Entropy quickly. All algorithms return exactly the same value for Sample Entropy, and no approximation techniques are used. We compare and evaluate them using cardiac inter-beat (RR) time series. We investigate three algorithms. The first one is an extension of the k d -trees algorithm, customized for Sample Entropy. The second one is an extension of an algorithm initially proposed for Approximate Entropy, again customized for Sample Entropy, but also improved to present even faster results. The last one is a completely new algorithm, presenting the fastest execution times for specific values of m, r, time series length, and signal characteristics. These algorithms are compared with the straightforward implementation, directly resulting from the definition of Sample Entropy, in order to give a clear image of the speedups achieved. All algorithms assume the classical approach to the metric, in which the maximum norm is used. The key idea of the two last suggested algorithms is to avoid unnecessary comparisons by detecting them early. We use the term unnecessary to refer to those comparisons for which we know a priori that they will fail at the similarity check. The number of avoided comparisons is proved to be very large, resulting in an analogous large reduction of execution time, making them the fastest algorithms available today for the computation of Sample Entropy.

Measuring body temperature time series regularity using approximate entropy and sample entropy

2009 ◽  
Author(s):  
D. Cuesta-Frau ◽  
P. Miro-Martinez ◽  
S. Oltra-Crespo ◽  
M. Varela-Entrecanales ◽  
M. Aboy ◽  
...  
Keyword(s):  
Time Series ◽  
Body Temperature ◽  
Approximate Entropy ◽  
Sample Entropy ◽  
Temperature Time Series ◽  
Temperature Time

Physiological time-series analysis using approximate entropy and sample entropy

2000 ◽  
Vol 278 (6) ◽  
pp. H2039-H2049 ◽  
Author(s):  
Joshua S. Richman ◽  
J. Randall Moorman
Keyword(s):  
Time Series ◽  
Approximate Entropy ◽  
Sample Entropy ◽  
Random Numbers ◽  
Data Sets ◽  
Production Methods ◽  
Biological Time ◽  
Biological Studies ◽  
Probabilistic Character ◽  
Improved Accuracy

Entropy, as it relates to dynamical systems, is the rate of information production. Methods for estimation of the entropy of a system represented by a time series are not, however, well suited to analysis of the short and noisy data sets encountered in cardiovascular and other biological studies. Pincus introduced approximate entropy (ApEn), a set of measures of system complexity closely related to entropy, which is easily applied to clinical cardiovascular and other time series. ApEn statistics, however, lead to inconsistent results. We have developed a new and related complexity measure, sample entropy (SampEn), and have compared ApEn and SampEn by using them to analyze sets of random numbers with known probabilistic character. We have also evaluated cross-ApEn and cross-SampEn, which use cardiovascular data sets to measure the similarity of two distinct time series. SampEn agreed with theory much more closely than ApEn over a broad range of conditions. The improved accuracy of SampEn statistics should make them useful in the study of experimental clinical cardiovascular and other biological time series.

scholarly journals New Fast ApEn and SampEn Entropy Algorithms Implementation and Their Application to Supercomputer Power Consumption

Entropy ◽  
2020 ◽  
Vol 22 (8) ◽  
pp. 863 ◽  
Author(s):  
Jiří Tomčala
Keyword(s):  
Time Series ◽  
Power Consumption ◽  
Real World ◽  
Software Package ◽  
World System ◽  
Approximate Entropy ◽  
R Package ◽  
Sample Entropy ◽  
R Software

Approximate Entropy and especially Sample Entropy are recently frequently used algorithms for calculating the measure of complexity of a time series. A lesser known fact is that there are also accelerated modifications of these two algorithms, namely Fast Approximate Entropy and Fast Sample Entropy. All these algorithms are effectively implemented in the R software package TSEntropies. This paper contains not only an explanation of all these algorithms, but also the principle of their acceleration. Furthermore, the paper contains a description of the functions of this software package and their parameters, as well as simple examples of using this software package to calculate these measures of complexity of an artificial time series and the time series of a complex real-world system represented by the course of supercomputer infrastructure power consumption. These time series were also used to test the speed of this package and to compare its speed with another R package pracma. The results show that TSEntropies is up to 100 times faster than pracma and another important result is that the computational times of the new Fast Approximate Entropy and Fast Sample Entropy algorithms are up to 500 times lower than the computational times of their original versions. At the very end of this paper, the possible use of this software package TSEntropies is proposed.

scholarly journals Relative Consistency of Sample Entropy Is Not Preserved in MIX Processes

Entropy ◽  
2020 ◽  
Vol 22 (6) ◽  
pp. 694
Author(s):  
Sebastian Żurek ◽  
Waldemar Grabowski ◽  
Klaudia Wojtiuk ◽  
Dorota Szewczak ◽  
Przemysław Guzik ◽  
...  
Keyword(s):  
Time Series ◽  
Approximate Entropy ◽  
Sample Entropy ◽  
Formal Proof ◽  
Economic Time Series ◽  
Seminal Paper ◽  
Relative Consistency ◽  
Economic Time ◽  
Central Characteristic ◽  
Stochastic Element

Relative consistency is a notion related to entropic parameters, most notably to Approximate Entropy and Sample Entropy. It is a central characteristic assumed for e.g., biomedical and economic time series, since it allows the comparison between different time series at a single value of the threshold parameter r. There is no formal proof for this property, yet it is generally accepted that it is true. Relative consistency in both Approximate Entropy and Sample entropy was first tested with the M I X process. In the seminal paper by Richman and Moorman, it was shown that Approximate Entropy lacked the property for cases in which Sample Entropy did not. In the present paper, we show that relative consistency is not preserved for M I X processes if enough noise is added, yet it is preserved for another process for which we define a sum of a sinusoidal and a stochastic element, no matter how much noise is present. The analysis presented in this paper is only possible because of the existence of the very fast NCM algorithm for calculating correlation sums and thus also Sample Entropy.

Cross-sample entropy estimation for time series analysis: a nonparametric approach

2021 ◽  
Author(s):  
Ignacio Ramírez-Parietti ◽  
Javier E. Contreras-Reyes ◽  
Byron J. Idrovo-Aguirre
Keyword(s):  
Time Series ◽  
Time Series Analysis ◽  
Sample Entropy ◽  
Entropy Estimation ◽  
Nonparametric Approach ◽  
Series Analysis

scholarly journals Multiscale Sample Entropy of Two-Dimensional Decaying Turbulence

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 245
Author(s):  
Ildoo Kim
Keyword(s):  
Time Series ◽  
Time Series Data ◽  
Soap Film ◽  
Sample Entropy ◽  
Series Data ◽  
Entropy Analysis ◽  
Measured Time ◽  
Energetic Analysis ◽  
Turbulence Data ◽  
Decaying Turbulence

Multiscale sample entropy analysis has been developed to quantify the complexity and the predictability of a time series, originally developed for physiological time series. In this study, the analysis was applied to the turbulence data. We measured time series data for the velocity fluctuation, in either the longitudinal or transverse direction, of turbulent soap film flows at various locations. The research was to assess the feasibility of using the entropy analysis to qualitatively characterize turbulence, without using any conventional energetic analysis of turbulence. The study showed that the application of the entropy analysis to the turbulence data is promising. From the analysis, we successfully captured two important features of the turbulent soap films. It is indicated that the turbulence is anisotropic from the directional disparity. In addition, we observed that the most unpredictable time scale increases with the downstream distance, which is an indication of the decaying turbulence.

scholarly journals The Systematic Bias of Entropy Calculation in the Multi-Scale Entropy Algorithm

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 659
Author(s):  
Jue Lu ◽  
Ze Wang
Keyword(s):  
Time Series ◽  
Standard Deviation ◽  
Coarse Graining ◽  
Sample Entropy ◽  
Systematic Bias ◽  
Original Time Series ◽  
Multi Scale ◽  
Calculation Results ◽  
Surrogate Measure ◽  
Original Time

Entropy indicates irregularity or randomness of a dynamic system. Over the decades, entropy calculated at different scales of the system through subsampling or coarse graining has been used as a surrogate measure of system complexity. One popular multi-scale entropy analysis is the multi-scale sample entropy (MSE), which calculates entropy through the sample entropy (SampEn) formula at each time scale. SampEn is defined by the “logarithmic likelihood” that a small section (within a window of a length m) of the data “matches” with other sections will still “match” the others if the section window length increases by one. “Match” is defined by a threshold of r times standard deviation of the entire time series. A problem of current MSE algorithm is that SampEn calculations at different scales are based on the same matching threshold defined by the original time series but data standard deviation actually changes with the subsampling scales. Using a fixed threshold will automatically introduce systematic bias to the calculation results. The purpose of this paper is to mathematically present this systematic bias and to provide methods for correcting it. Our work will help the large MSE user community avoiding introducing the bias to their multi-scale SampEn calculation results.

Sign in / Sign up

Export Citation Format

Share Document