Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society’s most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

Personalising the dilemma: research ethics in fiction

Learning about research ethics and research integrity is greatly facilitated by case studies, which illuminate, ground and personalise abstract questions. This paper argues that fiction can provide similar learning experiences, incarnating ethical dilemmas through a medium that is highly accessible yet sophisticated in its depictions of how researchers behave. Examples of fictional illustrations are given to illustrate various themes such as animal experimentation, exploitation of the vulnerable, researcher bias and research fraud.

Evaluation of Environmental Policy with Q Methodology

Environmental policy is often characterized by differences of opinion and polarized perceptions. This holds for all groups involved in lobbying, creating, implementing, and researching policy. Q methodology is a technique originally developed by William Stephenson in the 1930s for work in psychology as an alternative to R methodology, which was dominant at the time. R methodology involves gathering scores from subjects being analyzed, such as those generated by intelligence tests, and then correlating the scores with factors such as gender or ethnicity. Obviously, the scores are heavily dependent on the choice of questions set by the researcher in the tests. In contrast, Q methodology commonly uses statements generated by the participants of the study, and it is these that the subjects are asked to score. This helps to avoid the type of bias that might result from a researcher formulating the statements presented to the subjects, though it is important to note that researcher bias is also present in Q methodology through selection of the statements and the type of quantitative analysis used. In studies involving evaluation of environmental policy, Q methodology is typically used to elicit opinions from subjects by scoring participant statements obtained from interviews or statements from secondary sources such as written reports, news articles, or images. These scores are then correlated using factor analysis, and statements that group together are compiled to create discourses about different aspects of the environmental policy under evaluation.

The Oscillatory ReConstruction Algorithm adaptively identifies frequency bands to improve spectral decomposition in human and rodent neural recordings

Neural oscillations show substantial variability within and across individuals and brain regions, yet most existing studies analyze oscillations using canonical, fixed-frequency bands. Thus, there is an ongoing need for tools that capture oscillatory variability in neural signals. Toward this end, Oscillatory ReConstruction Algorithm is a novel and adaptive analytic tool that allows researchers to measure neural oscillations with more precision and less researcher bias.

Protecting against researcher bias in secondary data analysis: Challenges and solutions

Protecting against researcher biases – both conscious and unconscious – can help to ensure robust findings and correct inferences in epidemiology. While pre-registration can be an effective way to achieve this, it brings several challenges for researchers analysing existing datasets. Here we describe these challenges, and propose solutions and alternatives. For each solution, we provide guidance, and highlight practical considerations for researchers. The adoption of these practices will allow researchers to effectively pre-register secondary data analysis studies, or use an alternative approach, in order to protect themselves against common human biases. In turn, this will increase the robustness and credibility of epidemiological research based on secondary data.

May the power be with you: are there highly powered studies in neuroscience, and how can we get more of them?

Statistical power is essential for robust science and replicability, but a meta-analysis by Button et al. in 2013 diagnosed a “power failure” for neuroscience. In contrast, Nord et al. ( J Neurosci 37: 8051–8061, 2017) reanalyzed these data and suggested that some studies feature high power. We illustrate how publication and researcher bias might have inflated power estimates, and review recently introduced techniques that can improve analysis pipelines and increase power in neuroscience studies.