scholarly journals The comparison of two independent proportions via Design Analysis: Theoretical background and an R implementation

2021 ◽  
Author(s):  
Riccardo Bettin
Keyword(s):  
Statistical Power ◽  
Theoretical Background ◽  
Design Analysis ◽  
Small Sample ◽  
Questionable Research Practices ◽  
Replication Crisis ◽  
Long Time ◽  
Actual Publication ◽  
Small Sample Sizes ◽  
Post Hoc

In the last decade science has fallen into a replication crisis, this means that researches, when replicated, do not give the same results as the original ones. The difficulty in replicating studies can be due to several reasons, some of which regard the scientific world in general, such as the actual publication system that encourages incorrect behaviours and questionable research practices by scientists, and some that change between scientific fields. In fact, some scientific field feel this crisis more than others, and psychology is one of them. Low statistical power and misuse of statistics in psychology is reported from a long time. The first to criticize psychologist in regards of the use of power has been Cohen in 1962. This crisis can lead to the loss of trust in psychology and in science in general, for this reason it is important to find some solutions to the crisis.Several possible solutions have been proposed. In this work we will focus on Design analysis, that is quite a new notion that can help in the process of getting out of the replication crisis. This analysis consists in calculating (power, and) new type of inferential errors to help researchers to better understand the consequences of low power, small sample sizes and studies started without appropriate planning. Design analysis can be done prospectively and retrospectively, that is different from post-hoc power calculations. The aims of this work are mainly to extend the design analysis to the case of differences between independent proportions and to provide an R implementation that can be used by researchers.

scholarly journals G-computation and machine learning for estimating the causal effects of binary exposure statuses on binary outcomes

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Florent Le Borgne ◽  
Arthur Chatton ◽  
Maxime Léger ◽  
Rémi Lenain ◽  
Yohann Foucher
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Statistical Power ◽  
Small Sample ◽  
Causal Effects ◽  
Small Samples ◽  
Support Vector ◽  
Sample Sizes ◽  
Super Learner ◽  
Small Sample Sizes

AbstractIn clinical research, there is a growing interest in the use of propensity score-based methods to estimate causal effects. G-computation is an alternative because of its high statistical power. Machine learning is also increasingly used because of its possible robustness to model misspecification. In this paper, we aimed to propose an approach that combines machine learning and G-computation when both the outcome and the exposure status are binary and is able to deal with small samples. We evaluated the performances of several methods, including penalized logistic regressions, a neural network, a support vector machine, boosted classification and regression trees, and a super learner through simulations. We proposed six different scenarios characterised by various sample sizes, numbers of covariates and relationships between covariates, exposure statuses, and outcomes. We have also illustrated the application of these methods, in which they were used to estimate the efficacy of barbiturates prescribed during the first 24 h of an episode of intracranial hypertension. In the context of GC, for estimating the individual outcome probabilities in two counterfactual worlds, we reported that the super learner tended to outperform the other approaches in terms of both bias and variance, especially for small sample sizes. The support vector machine performed well, but its mean bias was slightly higher than that of the super learner. In the investigated scenarios, G-computation associated with the super learner was a performant method for drawing causal inferences, even from small sample sizes.

Distinguishing autism from co-existing conditions: a behavioural profiling investigation

2016 ◽  
Vol 2 (1) ◽  
pp. 41-54
Author(s):  
Ashleigh Saunders ◽  
Karen E. Waldie
Keyword(s):  
Statistical Power ◽  
Psychiatric Comorbidity ◽  
Adaptive Functioning ◽  
Autism Spectrum ◽  
Small Sample ◽  
Comorbid Conditions ◽  
Sample Sizes ◽  
Content Type ◽  
Small Sample Sizes ◽  
Comorbid Adhd

Purpose – Autism spectrum disorder (ASD) is a lifelong neurodevelopmental condition for which there is no known cure. The rate of psychiatric comorbidity in autism is extremely high, which raises questions about the nature of the co-occurring symptoms. It is unclear whether these additional conditions are true comorbid conditions, or can simply be accounted for through the ASD diagnosis. The paper aims to discuss this issue. Design/methodology/approach – A number of questionnaires and a computer-based task were used in the current study. The authors asked the participants about symptoms of ASD, attention deficit hyperactivity disorder (ADHD) and anxiety, as well as overall adaptive functioning. Findings – The results demonstrate that each condition, in its pure form, can be clearly differentiated from one another (and from neurotypical controls). Further analyses revealed that when ASD occurs together with anxiety, anxiety appears to be a separate condition. In contrast, there is no clear behavioural profile for when ASD and ADHD co-occur. Research limitations/implications – First, due to small sample sizes, some analyses performed were targeted to specific groups (i.e. comparing ADHD, ASD to comorbid ADHD+ASD). Larger sample sizes would have given the statistical power to perform a full scale comparative analysis of all experimental groups when split by their comorbid conditions. Second, males were over-represented in the ASD group and females were over-represented in the anxiety group, due to the uneven gender balance in the prevalence of these conditions. Lastly, the main profiling techniques used were questionnaires. Clinical interviews would have been preferable, as they give a more objective account of behavioural difficulties. Practical implications – The rate of psychiatric comorbidity in autism is extremely high, which raises questions about the nature of the co-occurring symptoms. It is unclear whether these additional conditions are true comorbid conditions, or can simply be accounted for through the ASD diagnosis. Social implications – This information will be important, not only to healthcare practitioners when administering a diagnosis, but also to therapists who need to apply evidence-based treatment to comorbid and stand-alone conditions. Originality/value – This study is the first to investigate the nature of co-existing conditions in ASD in a New Zealand population.

The Identity Status Continuum Revisited

2018 ◽  
Vol 23 (4) ◽  
pp. 289-299 ◽  
Author(s):  
Wim Meeus
Keyword(s):  
Identity Development ◽  
Identity Formation ◽  
Statistical Power ◽  
Developmental Psychology ◽  
Meta Analysis ◽  
Small Sample ◽  
Identity Status ◽  
Identity Statuses ◽  
Identity Maintenance ◽  
Small Sample Sizes

Abstract. The developmental continuum of identity status has been a topic of theoretical debate since the early 1980’s. A recent meta-analysis and recent studies with dual cycle models lead to two conclusions: (1) during adolescence there is systematic identity maturation; (2) there are two continuums of identity status progression. Both continuums show that in general adolescents move from transient identity statuses to identity statuses that mark the relative endpoints of development: from diffusion to closure, and from searching moratorium and moratorium to closure and achievement. This pattern can be framed as development from identity formation to identity maintenance. In Identity Status Interview research using Marcia’s model, not the slightest indication for a continuum of identity development was found. This may be due to the small sample sizes of the various studies leading to small statistical power to detect differences in identity status transitions, as well as developmental inconsistencies in Marcia’s model. Findings from this review are interpreted in terms of life-span developmental psychology.

Growth Need Strength and Context Satisfactions as Moderators of the Relations of the Job Characteristics Model

1992 ◽  
Vol 18 (3) ◽  
pp. 575-593 ◽  
Author(s):  
Robert B. Tiegs ◽  
Lois E. Tetrick ◽  
Yitzhak Fried
Keyword(s):  
Statistical Power ◽  
Job Characteristics ◽  
Job Security ◽  
Small Sample ◽  
Work Outcomes ◽  
Moderating Effects ◽  
Psychological States ◽  
Job Characteristics Model ◽  
Small Sample Sizes ◽  
Overall Job Satisfaction

Empirical investigations of the job characteristics model (JCM; Hackman & Oldham, 1980) have failed to systematically explore the moderating effects of growth need strength (GNS) and context satisfactions (viz., pay, job security, co-worker, and supervision) on the relations among the core job characteristics, critical psychological states, and work outcomes. Previous studies also are criticized for the use of subgroup analytic techniques, low statistical power resulting from small sample sizes (i.e, often less than 200) and/or samples consisting of individuals of relatively homogeneous jobs/occupations. As an attempt to address these deficiencies in the literature, this study examined the moderating effects of GNS and each of the four context satisfactions using a large sample (N = 6405) of employees from a variety of jobs and occupations. Overall, the results of univariate and multivariate hierarchical moderated multiple regression analyses suggest that none of thefive individual difference factors appeared to be viable moderators of any of the relations among job characteristics, psychological states, and three work outcomes (viz., growth satisfaction, overall job satisfaction, and internal motivation). Also, there was no supportive evidence for potential joint moderating effects between GNS and each context satisfaction on the relations of the JCM.

Statistics for Small Groups: The Power of the Pretest

1988 ◽  
Vol 13 (3) ◽  
pp. 142-146 ◽  
Author(s):  
David A. Cole
Keyword(s):  
Statistical Power ◽  
Change Score ◽  
Small Sample ◽  
Analysis Of Covariance ◽  
Small Samples ◽  
Group Differences ◽  
Control Group ◽  
Small Sample Sizes ◽  
Almost All ◽  
Treatment Control Group

In the area of severe-profound retardation, researchers are faced with small sample sizes. The question of statistical power is critical. In this article, three commonly used tests for treatment-control group differences are compared with respect to their relative power: the posttest-only approach, the change-score approach, and an analysis of covariance (ANCOVA) approach. In almost all cases, the ANCOVA approach is the more powerful than the other two, even when very small samples are involved. Finally, a fourth approach involving ANCOVA plus alternate rank assignments is examined and found to be superior even to the ANCOVA approach, especially in small sample cases. Use of slightly more sophisticated statistics in small sample research is recommended.

scholarly journals The inheritance of metabolic flux: expressions for the within-sibship mean and variance given the parental genotypes.

Genetics ◽  
1990 ◽  
Vol 125 (3) ◽  
pp. 655-667
Author(s):  
P J Ward
Keyword(s):  
Statistical Power ◽  
Metabolic Flux ◽  
Genetic Variance ◽  
Small Sample ◽  
Polygenic Inheritance ◽  
Allozymic Variation ◽  
Recent Developments ◽  
Mean And Variance ◽  
Small Sample Sizes ◽  
Adequate Power

Abstract Recent developments have related quantitative trait expression to metabolic flux. The present paper investigates some implications of this for statistical aspects of polygenic inheritance. Expressions are derived for the within-sibship genetic mean and genetic variance of metabolic flux given a pair of parental, diploid, n-locus genotypes. These are exact and hold for arbitrary numbers of gene loci, arbitrary allelic values at each locus, and for arbitrary recombination fractions between adjacent gene loci. The within-sibship, genetic variance is seen to be simply a measure of parental heterozygosity plus a measure of the degree of linkage coupling within the parental genotypes. Approximations are given for the within-sibship phenotypic mean and variance of metabolic flux. These results are applied to the problem of attaining adequate statistical power in a test of association between allozymic variation and inter-individual variation in metabolic flux. Simulations indicate that statistical power can be greatly increased by augmenting the data with predictions and observations on progeny statistics in relation to parental allozyme genotypes. Adequate power may thus be attainable at small sample sizes, and when allozymic variation is scored at a only small fraction of the total set of loci whose catalytic products determine the flux.

scholarly journals Meta-Analysis of Microdissected Breast Tumors Reveals Genes Regulated in the Stroma but Hidden in Bulk Analysis

2021 ◽  
Author(s):  
Aurora Savino ◽  
Niccolò de Marzo ◽  
Paolo Provero ◽  
Valeria Poli
Keyword(s):  
Statistical Power ◽  
Molecular Mechanisms ◽  
Meta Analysis ◽  
Breast Tumors ◽  
Small Sample ◽  
Bulk Analysis ◽  
Small Sample Sizes ◽  
Microarray Datasets ◽  
User Friendly ◽  
Cellular Components

Background: transcriptome data provide a valuable resource for the study of cancer molecular mechanisms, but technical biases, samples’ heterogeneity and small sample sizes result in poorly reproducible lists of regulated genes. Additionally, the presence of multiple cellular components contributing to cancer development complicate the interpretation of bulk transcriptomic profiles. Methods: we collected 48 microarray datasets of laser capture microdissected breast tumors, and performed a meta-analysis to identify robust lists of genes differentially expressed in these tumors. We created a database with carefully harmonized metadata to be used as a resource for the research community. Results: combining the results of multiple datasets improved the statistical power, and the analysis of stroma and epithelium separately allows identifying genes with different contribution in each compartment. Conclusions: our database can profitably help biomarkers’ discovery and is readily accessible through a user-friendly web interface (https://aurorasavino.shinyapps.io/metalcm/).

scholarly journals Breaking the circularity in circular analyses: Simulations and formal treatment of the flattened average approach

2020 ◽  
Vol 16 (11) ◽  
pp. e1008286
Author(s):  
Howard Bowman ◽  
Joseph L. Brooks ◽  
Omid Hajilou ◽  
Alexia Zoumpoulaki ◽  
Vladimir Litvak
Keyword(s):  
Statistical Power ◽  
Type I Error ◽  
False Positive Rate ◽  
Error Rates ◽  
Type I ◽  
Mathematical Proofs ◽  
Positive Rate ◽  
Replication Crisis ◽  
Post Hoc ◽  
Type Ii Errors

There has been considerable debate and concern as to whether there is a replication crisis in the scientific literature. A likely cause of poor replication is the multiple comparisons problem. An important way in which this problem can manifest in the M/EEG context is through post hoc tailoring of analysis windows (a.k.a. regions-of-interest, ROIs) to landmarks in the collected data. Post hoc tailoring of ROIs is used because it allows researchers to adapt to inter-experiment variability and discover novel differences that fall outside of windows defined by prior precedent, thereby reducing Type II errors. However, this approach can dramatically inflate Type I error rates. One way to avoid this problem is to tailor windows according to a contrast that is orthogonal (strictly parametrically orthogonal) to the contrast being tested. A key approach of this kind is to identify windows on a fully flattened average. On the basis of simulations, this approach has been argued to be safe for post hoc tailoring of analysis windows under many conditions. Here, we present further simulations and mathematical proofs to show exactly why the Fully Flattened Average approach is unbiased, providing a formal grounding to the approach, clarifying the limits of its applicability and resolving published misconceptions about the method. We also provide a statistical power analysis, which shows that, in specific contexts, the fully flattened average approach provides higher statistical power than Fieldtrip cluster inference. This suggests that the Fully Flattened Average approach will enable researchers to identify more effects from their data without incurring an inflation of the false positive rate.

Functional Specificity and Sex Differences in the Neural Circuits Supporting the Inhibition of Automatic Imitation

2018 ◽  
Author(s):  
Kohinoor Monish Darda ◽  
Emily E. Butler ◽  
Richard Ramsey
Keyword(s):  
Sex Differences ◽  
Statistical Power ◽  
Small Sample ◽  
Neural Systems ◽  
Specific System ◽  
Analysis Pipeline ◽  
Automatic Imitation ◽  
Domain Specific ◽  
Neural Underpinnings ◽  
Small Sample Sizes

Although humans show an involuntary tendency to copy other people’s actions, which builds rapport between individuals, we do not copy actions indiscriminately. Instead, copying behaviours are guided by a selection mechanism, which inhibits some actions and prioritises others. To date, the neural underpinnings of the inhibition of automatic imitation and differences between the sexes in imitation control are not well understood. Previous studies involved small sample sizes and low statistical power, which produced mixed findings regarding the involvement of domain-general and domain-specific neural architectures. Here, we used data from Experiment 1 (N=28) to perform a power analysis to determine the sample size required for Experiment 2 (N=50; 80% power). Using independent functional localisers and an analysis pipeline that bolsters sensitivity, during imitation control we show clear engagement of the multiple-demand network (domain-general), but no sensitivity in the theory-of-mind network (domain-specific). Weaker effects were observed with regard to sex differences, suggesting that there are more similarities than differences between the sexes in terms of the neural systems engaged during imitation control. In sum, neurocognitive models of imitation require revision to reflect that the inhibition of imitation relies on a domain-general selection system rather than a domain-specific system supporting social cognition.

scholarly journals Indices of Rank Histogram Flatness and Their Sampling Properties

2019 ◽  
Vol 147 (2) ◽  
pp. 763-769 ◽  
Author(s):  
D. S. Wilks
Keyword(s):  
Null Hypothesis ◽  
Statistical Power ◽  
Small Sample ◽  
Sample Sizes ◽  
Sampling Distributions ◽  
Power Of Tests ◽  
Rank Histogram ◽  
Small Sample Sizes ◽  
Formal Hypothesis Testing ◽  
Two Alternatives

Abstract Quantitative evaluation of the flatness of the verification rank histogram can be approached through formal hypothesis testing. Traditionally, the familiar χ2 test has been used for this purpose. Recently, two alternatives—the reliability index (RI) and an entropy statistic (Ω)—have been suggested in the literature. This paper presents approximations to the sampling distributions of these latter two rank histogram flatness metrics, and compares the statistical power of tests based on the three statistics, in a controlled setting. The χ2 test is generally most powerful (i.e., most sensitive to violations of the null hypothesis of rank uniformity), although for overdispersed ensembles and small sample sizes, the test based on the entropy statistic Ω is more powerful. The RI-based test is preferred only for unbiased forecasts with small ensembles and very small sample sizes.

Sign in / Sign up

Export Citation Format

Share Document