Biostatistics for Multiple Testing

Multiple testings are instances that contain simultaneous tests for more than one hypothesis. When multiple testings are conducted at the same time, it is more likely that the null hypothesis is rejected, even if the null hypothesis is correct. If individual hypothesis decisions are based on unadjusted <i>p</i>-values, it is usually more likely that some of the true null hypotheses will be rejected. In order to solve the multiple testing problems, various studies have attempted to increase the power by taking into account the family-wise error rate or false discovery rate and statistics required for testing hypotheses. This article discuss methods that account for the multiplicity issue and introduces various statistical techniques.

Estimating the false finding rate across scientific fields

The possiblity of large amounts of false positives within the scientific literature has gained significant attention, particular in light of several replication projects in which large proportions of published studies have failed to replicate. I show through simulation that low replication rates can occur even when the published literature contains mostly true (non-null) findings. Using conservative estimates of the proportion of true null hypotheses within published studies based on replications, I show that the results of recent replication projects are consistent with the possiblity that most published research is true.

Estimating the Proportion of True Null Hypotheses in Multiple Testing Problems

The problem of estimating the proportion, π0, of the true null hypotheses in a multiple testing problem is important in cases where large scale parallel hypotheses tests are performed independently. While the problem is a quantity of interest in its own right in applications, the estimate of π0 can be used for assessing or controlling an overall false discovery rate. In this article, we develop an innovative nonparametric maximum likelihood approach to estimate π0. The nonparametric likelihood is proposed to be restricted to multinomial models and an EM algorithm is also developed to approximate the estimate of π0. Simulation studies show that the proposed method outperforms other existing methods. Using experimental microarray datasets, we demonstrate that the new method provides satisfactory estimate in practice.