scholarly journals Advice on comparing two independent samples of circular data in biology

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Lukas Landler ◽  
Graeme D. Ruxton ◽  
E. Pascal Malkemper
Keyword(s):  
Error Rate ◽  
Statistical Power ◽  
Type I Error ◽  
Statistical Treatment ◽  
Good Control ◽  
Circular Data ◽  
Type I ◽  
Trigonometric Functions ◽  
Biological Variables ◽  
Two Samples

AbstractMany biological variables are recorded on a circular scale and therefore need different statistical treatment. A common question that is asked of such circular data involves comparison between two groups: Are the populations from which the two samples are drawn differently distributed around the circle? We compared 18 tests for such situations (by simulation) in terms of both abilities to control Type-I error rate near the nominal value, and statistical power. We found that only eight tests offered good control of Type-I error in all our simulated situations. Of these eight, we were able to identify the Watson’s U2 test and a MANOVA approach, based on trigonometric functions of the data, as offering the best power in the overwhelming majority of our test circumstances. There was often little to choose between these tests in terms of power, and no situation where either of the remaining six tests offered substantially better power than either of these. Hence, we recommend the routine use of either Watson’s U2 test or MANOVA approach when comparing two samples of circular data.

scholarly journals Comparing two circular distributions: advice for effective implementation of statistical procedures in biology

2021 ◽  
Author(s):  
Lukas Landler ◽  
Graeme D Ruxton ◽  
Erich Pascal Malkemper
Keyword(s):  
Statistical Power ◽  
Type I Error ◽  
Statistical Treatment ◽  
Good Control ◽  
Time Of Day ◽  
Circular Data ◽  
Type I ◽  
Biological Variables ◽  
Circular Distributions ◽  
Two Samples

Many biological variables, often involving timings of events or directions, are recorded on a circular rather than linear scale, and need different statistical treatment for that reason. A common question that is asked of such circular data involves comparison between two groups or treatments: Are the populations from which the two samples drawn differently distributed around the circle? For example, we might ask whether the distribution of directions from which a stalking predator approaches its prey differs between sunny and cloudy conditions; or whether the time of day of mating attempts differs between lab mice subject to one of two hormone treatments. An array of statistical approaches to these questions have been developed. We compared 18 of these (by simulation) in terms of both abilities to control type I error rate near the nominal value, and statistical power. We found that only eight tests offered good control of type I error in all our test situations. Of these eight, we are able to identify Watsons U^2 test and MANOVA based on trigonometric functions of the data as offering the best power in the overwhelming majority of our test circumstances. There was often little to choose between these tests in terms of power, and no situation where either of the remaining six tests offered substantially better power than either of these. Hence, we recommend the routine use of either Watsons U^2 test or MANOVA when comparing two samples of circular data.

A more efficient three-arm non-inferiority test based on pooled estimators of the homogeneous variance

2016 ◽  
Vol 27 (8) ◽  
pp. 2437-2446 ◽  
Author(s):  
Hezhi Lu ◽  
Hua Jin ◽  
Weixiong Zeng
Keyword(s):  
Sample Size ◽  
Error Rate ◽  
Statistical Power ◽  
Type I Error ◽  
Statistical Testing ◽  
New Method ◽  
Type I ◽  
Simulation Studies ◽  
Testing Framework ◽  
Better Than

Hida and Tango established a statistical testing framework for the three-arm non-inferiority trial including a placebo with a pre-specified non-inferiority margin to overcome the shortcomings of traditional two-arm non-inferiority trials (such as having to choose the non-inferiority margin). In this paper, we propose a new method that improves their approach with respect to two aspects. We construct our testing statistics based on the best unbiased pooled estimators of the homogeneous variance; and we use the principle of intersection-union tests to determine the rejection rule. We theoretically prove that our test is better than that of Hida and Tango for large sample sizes. Furthermore, when that sample size was small or moderate, our simulation studies showed that our approach performed better than Hida and Tango’s. Although both controlled the type I error rate, their test was more conservative and the statistical power of our test was higher.

scholarly journals Model selection versus traditional hypothesis testing in circular statistics: a simulation study

Biology Open ◽  
2020 ◽  
Vol 9 (6) ◽  
pp. bio049866
Author(s):  
Lukas Landler ◽  
Graeme D. Ruxton ◽  
E. Pascal Malkemper
Keyword(s):  
Type I Error ◽  
Statistical Treatment ◽  
Model Fitting ◽  
Standard Technique ◽  
Circular Data ◽  
Circular Statistics ◽  
Distribution Model ◽  
Type I ◽  
Rayleigh Test ◽  
Made In

ABSTRACTMany studies in biology involve data measured on a circular scale. Such data require different statistical treatment from those measured on linear scales. The most common statistical exploration of circular data involves testing the null hypothesis that the data show no aggregation and are instead uniformly distributed over the whole circle. The most common means of performing this type of investigation is with a Rayleigh test. An alternative might be to compare the fit of the uniform distribution model to alternative models. Such model-fitting approaches have become a standard technique with linear data, and their greater application to circular data has been recently advocated. Here we present simulation data that demonstrate that such model-based inference can offer very similar performance to the best traditional tests, but only if adjustment is made in order to control type I error rate.

scholarly journals Likelihood-based Tests for Detecting Circadian Rhythmicity and Differential Circadian Patterns in Transcriptomic Applications

2021 ◽  
Author(s):  
Haocheng Ding ◽  
Lingsong Meng ◽  
Andrew C. Liu ◽  
Michelle L. Gumz ◽  
Andrew J. Bryant ◽  
...  
Keyword(s):  
Error Rate ◽  
Statistical Power ◽  
Type I Error ◽  
Brain Aging ◽  
Circadian Rhythmicity ◽  
Postmortem Brain ◽  
Superior Performance ◽  
Type I ◽  
Type I Error Rate ◽  
Circadian Patterns

AbstractCircadian rhythmicity in transcriptomic profiles has been shown in many physiological processes, and the disruption of circadian patterns has been founded to associate with several diseases. In this paper, we developed a series of likelihood-based methods to detect (i) circadian rhythmicity (denoted as LR rhythmicity) and (ii) differential circadian patterns comparing two experimental conditions (denoted as LR diff). In terms of circadian rhythmicity detection, we demonstrated that our proposed LR rhythmicity could better control the type I error rate compared to existing methods under a wide variety of simulation settings. In terms of differential circadian patterns, we developed methods in detecting differential amplitude, differential phase, differential basal level, and differential fit, which also successfully controlled the type I error rate. In addition, we demonstrated that the proposed LR diff could achieve higher statistical power in detecting differential fit, compared to existing methods. The superior performance of LR rhythmicity and LR diff was demonstrated in two real data applications, including a brain aging data (gene expression microarray data of human postmortem brain) and a time-restricted feeding data (RNA sequencing data of human skeletal muscles). An R package for our methods is publicly available on GitHub https://github.com/diffCircadian/diffCircadian.

scholarly journals Design and Statistical Analysis of Pooled Next Generation Sequencing for Rare Variants

2012 ◽  
Vol 2012 ◽  
pp. 1-19 ◽  
Author(s):  
Tao Wang ◽  
Chang-Yun Lin ◽  
Yuanhao Zhang ◽  
Ruofeng Wen ◽  
Kenny Ye
Keyword(s):  
Next Generation Sequencing ◽  
Error Rate ◽  
Statistical Power ◽  
Rare Variants ◽  
Good Control ◽  
Testing Procedure ◽  
Sequencing Error ◽  
Type I ◽  
Next Generation ◽  
Generation Sequencing

Next generation sequencing (NGS) is a revolutionary technology for biomedical research. One highly cost-efficient application of NGS is to detect disease association based on pooled DNA samples. However, several key issues need to be addressed for pooled NGS. One of them is the high sequencing error rate and its high variability across genomic positions and experiment runs, which, if not well considered in the experimental design and analysis, could lead to either inflated false positive rates or loss in statistical power. Another important issue is how to test association of a group of rare variants. To address the first issue, we proposed a new blocked pooling design in which multiple pools of DNA samples from cases and controls are sequenced together on same NGS functional units. To address the second issue, we proposed a testing procedure that does not require individual genotypes but by taking advantage of multiple DNA pools. Through a simulation study, we demonstrated that our approach provides a good control of the type I error rate, and yields satisfactory power compared to the test-based on individual genotypes. Our results also provide guidelines for designing an efficient pooled.

scholarly journals Hello again, ANOVA: rethinking ANOVA in the context of confirmatory data analysis

2021 ◽  
Author(s):  
Haiyang Jin
Keyword(s):  
Data Analysis ◽  
Error Rate ◽  
Statistical Power ◽  
Type I Error ◽  
Interaction Analysis ◽  
Type I ◽  
Type I Error Rate ◽  
Exploratory Approach ◽  
Confirmatory Data Analysis ◽  
Selection Of

Analysis of variance (ANOVA) is one of the most popular statistical methods employed for data analysis in psychology and other fields. Nevertheless, ANOVA is frequently used as an exploratory approach, even in confirmatory studies with explicit hypotheses. Such misapplication may invalidate ANOVA conventions, resulting in reduced statistical power, and even threatening the validity of conclusions. This paper evaluates the appropriateness of ANOVA conventions, discusses the potential motivations possibly misunderstood by researchers, and provides practical suggestions. Moreover, this paper proposes to control the Type I error rate with Hypothesis-based Type I Error Rate to consider both the number of tests and their logical relationships in rejecting the null hypothesis. Furthermore, this paper introduces the simple interaction analysis, which can employ the most straightforward interaction to test a hypothesis of interest. Finally, pre-registration is recommended to provide clarity for the selection of appropriate ANOVA tests in both confirmatory and exploratory studies.

scholarly journals Benefits of Open and High-Powered Research Outweigh Costs

2016 ◽  
Author(s):  
Etienne P. LeBel ◽  
Lorne Campbell ◽  
Timothy Loving
Keyword(s):  
Error Rate ◽  
Statistical Power ◽  
Type I Error ◽  
Open Data ◽  
Open Science ◽  
Type I ◽  
Incentive Structure ◽  
Science Practices ◽  
Type I Error Rate ◽  
Inflated Type

Several researchers recently outlined unacknowledged costs of open science practices, arguing these costs may outweigh benefits and stifle discovery of novel findings. We scrutinize these researchers' (1) statistical concern that heightened stringency with respect to false-positives will increase false-negatives and (2) meta-scientific concern that larger samples and executing direct replications engender opportunity costs that will decrease the rate of making novel discoveries. We argue their statistical concern is unwarranted given open science proponents recommend such practices to reduce the inflated Type I error rate from .35 down to .05 and simultaneously call for high-powered research to reduce the inflated Type II error rate. Regarding their meta-concern, we demonstrate that incurring some costs is required to increase the rate (and frequency) of making true discoveries because distinguishing true from false hypotheses requires a low Type I error rate, high statistical power, and independent direct replications. We also examine pragmatic concerns raised regarding adopting open science practices for relationship science (pre-registration, open materials, open data, direct replications, sample size); while acknowledging these concerns, we argue they are overstated given available solutions. We conclude benefits of open science practices outweigh costs for both individual researchers and the collective field in the long run, but that short term costs may exist for researchers because of the currently dysfunctional academic incentive structure. Our analysis implies our field's incentive structure needs to change whereby better alignment exists between researcher's career interests and the field's cumulative progress. We delineate recent proposals aimed at such incentive structure re-alignment.

scholarly journals Application of the Hierarchical Bootstrap to Multi-Level Data in Neuroscience

2019 ◽  
Author(s):  
Varun Saravanan ◽  
Gordon J. Berman ◽  
Samuel J. Sober
Keyword(s):  
Error Rate ◽  
Statistical Power ◽  
Type I Error ◽  
Statistical Tests ◽  
False Positive Rate ◽  
Type I ◽  
Type I Error Rate ◽  
The Hierarchical Structure ◽  
Positive Rate ◽  
Student’S T

AbstractA common feature in many neuroscience datasets is the presence of hierarchical data structures, most commonly recording the activity of multiple neurons in multiple animals across multiple trials. Accordingly, the measurements constituting the dataset are not independent, even though the traditional statistical analyses often applied in such cases (e.g. Student’s t-test) treat them as such. The hierarchical bootstrap has been shown to be an effective tool to accurately analyze such data and while it has been used extensively in the statistical literature, its use is not widespread in neuroscience - despite the ubiquity of hierarchical datasets. In this paper, we illustrate the intuitiveness and utility of this approach to analyze hierarchically nested datasets. We use simulated neural data to show that traditional statistical tests can result in a false positive rate of over 45%, even if the Type-I error rate is set at 5%. While summarizing data across non-independent points (or lower levels) can potentially fix this problem, this approach greatly reduces the statistical power of the analysis. The hierarchical bootstrap, when applied sequentially over the levels of the hierarchical structure, keeps the Type-I error rate within the intended bound and retains more statistical power than summarizing methods. We conclude by demonstrating the effectiveness of the method in two real-world examples, first analyzing singing data in male Bengalese finches (Lonchura striata var. domestica) and second quantifying changes in behavior under optogenetic control in flies (Drosophila melanogaster).

scholarly journals Bootstrat: Population Informed Bootstrapping for Rare Variant Tests

2016 ◽  
Author(s):  
Hailiang Huang ◽  
Gina M. Peloso ◽  
Daniel Howrigan ◽  
Barbara Rakitsch ◽  
Carl Johann Simon-Gabriel ◽  
...  
Keyword(s):  
Rare Variant ◽  
Error Rate ◽  
Population Stratification ◽  
Statistical Power ◽  
Type I Error ◽  
Rare Variants ◽  
Null Distribution ◽  
Population Heterogeneity ◽  
Type I ◽  
Testing Methods

AbstractRecent advances in genotyping and sequencing technologies have made detecting rare variants in large cohorts possible. Various analytic methods for associating disease to rare variants have been proposed, including burden tests, C-alpha and SKAT. Most of these methods, however, assume that samples come from a homogeneous population, which is not realistic for analyses of large samples. Not correcting for population stratification causes inflated p-values and false-positive associations. Here we propose a population-informed bootstrap resampling method that controls for population stratification (Bootstrat) in rare variant tests. In essence, the Bootstrat procedure uses genetic distance to create a phenotype probability for each sample. We show that this empirical approach can effectively correct for population stratification while maintaining statistical power comparable to established methods of controlling for population stratification. The Bootstrat scheme can be easily applied to existing rare variant testing methods with reasonable computational complexity.Author SummaryRecent technology advances have enabled large-scale analysis of rare variants, but properly testing rare variants remains a significant challenge as most rare variant testing methods assume a sample of homogenous ethnicity, an assumption often not true for large cohorts. Failure to account for this heterogeneity increases the type I error rate. Here we propose a bootstrap scheme applicable to most existing rare variant testing methods to control for population heterogeneity. This scheme uses a randomization layer to establish a null distribution of the test statistics while preserving the sample genetic relationships. The null distribution is then used to calculate an empirical p-value that accounts for population heterogeneity. We demonstrate how this scheme successfully controls the type I error rate without loss of statistical power.

How to Detect Publication Bias in Psychological Research

2019 ◽  
Vol 227 (4) ◽  
pp. 261-279 ◽  
Author(s):  
Frank Renkewitz ◽  
Melanie Keiner
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Type I Error ◽  
Psychological Research ◽  
Type I ◽  
True Effect Size ◽  
Questionable Research Practices ◽  
True Effect ◽  
Meta Analyses

Abstract. Publication biases and questionable research practices are assumed to be two of the main causes of low replication rates. Both of these problems lead to severely inflated effect size estimates in meta-analyses. Methodologists have proposed a number of statistical tools to detect such bias in meta-analytic results. We present an evaluation of the performance of six of these tools. To assess the Type I error rate and the statistical power of these methods, we simulated a large variety of literatures that differed with regard to true effect size, heterogeneity, number of available primary studies, and sample sizes of these primary studies; furthermore, simulated studies were subjected to different degrees of publication bias. Our results show that across all simulated conditions, no method consistently outperformed the others. Additionally, all methods performed poorly when true effect sizes were heterogeneous or primary studies had a small chance of being published, irrespective of their results. This suggests that in many actual meta-analyses in psychology, bias will remain undiscovered no matter which detection method is used.

Sign in / Sign up

Export Citation Format

Share Document