scholarly journals Methodological quality assessment in systematic reviews in health sciences that included observational studies: a cross-sectional study/ Avaliação da qualidade metodológica em revisões sistemáticas na área das ciências da saúde que incluíram estudos observacionais: um estudo transversal

2021 ◽  
Vol 4 (5) ◽  
pp. 23002-23018
Author(s):  
Carla Massignan ◽  
Luciana Butini Oliveira ◽  
Lia Rosana Honnef ◽  
Júlia Meller de Oliveira ◽  
Karyn Lehmkuhl ◽  
...  

There is great variability in methodological quality assessment (MQ) systematic reviews (SRs). We identified how MQ assessment in SRs that included observational studies in Health Science is applied and explored associated characteristics. A search was conducted in PubMed and five trained reviewers randomly selected 1,025 references after sample size calculation. Only SR published in English (September 2019/2020) were included. Selection and data extraction were conducted in two phases. Data were analyzed descriptively and using logistic regressions. After eligibility criteria application, 205 SRs were included. Only 27.8% informed the protocol registration and 80.0% described having followed a reporting guideline. Proportion’ SRs did not seem to present MQ assessment (OR 4.22; 95% CI: 1.38-12.87; P =0.01). SRs that did not register the study protocol (9.70; 95% CI: 1.95-48.27; P 0.001), those that did not inform the included study design (5.96; 95% CI: 1.63-21.77; P 0.001) and those without MA (OR 8.90; 95% CI: 2.79-28.43; p0.001) increased the odds of not evaluating MQ. Newcastle-Ottawa (47.5%), Joanna Briggs Institute (8.2%) and National Institute of Health (7.1%) were the most commonly used MQ tools. Lack of protocol registration, absence of information about the design of included studies, absence of MA, and proportion’ SRs were associated with lack of MQ assessment.

BMJ Open ◽  
2019 ◽  
Vol 9 (11) ◽  
pp. e028238 ◽  
Author(s):  
Shimels Hussien Mohammed ◽  
Tesfa Dejenie Habtewold ◽  
Mulugeta Molla Birhanu ◽  
Tesfamichael Awoke Sissay ◽  
Balewgizie Sileshi Tegegne ◽  
...  

ObjectiveLow neighbourhood socioeconomic status (NSES) has been linked to a higher risk of overweight/obesity, irrespective of the individual’s own socioeconomic status. No meta-analysis study has been done on the association. Thus, this study was done to synthesise the existing evidence on the association of NSES with overweight, obesity and body mass index (BMI).DesignSystematic review and meta-analysis.Data sourcesPubMed, Embase, Scopus, Cochrane Library, Web of Sciences and Google Scholar databases were searched for articles published until 25 September 2019.Eligibility criteriaEpidemiological studies, both longitudinal and cross-sectional ones, which examined the link of NSES to overweight, obesity or BMI, were included.Data extraction and synthesisData extraction was done by two reviewers, working independently. The methodological quality of included studies was assessed using the Newcastle-Ottawa Scale for the observational studies. The summary estimates of the relationships of NSES with overweight, obesity and BMI statuses were calculated with random-effects meta-analysis models. Heterogeneity was assessed by Cochran’s Q and I2 statistics. Subgroup analyses were done by age categories, continents, study designs and NSES measures. Publication bias was assessed by visual inspection of funnel plots and Egger’s regression test.ResultA total of 21 observational studies, covering 1 244 438 individuals, were included in this meta-analysis. Low NSES, compared with high NSES, was found to be associated with a 31% higher odds of overweight (pooled OR 1.31, 95% CI 1.16 to 1.47, p<0.001), a 45% higher odds of obesity (pooled OR 1.45, 95% CI 1.21 to 1.74, p<0.001) and a 1.09 kg/m2 increase in mean BMI (pooled beta=1.09, 95% CI 0.67 to 1.50, p<0.001).ConclusionNSES disparity might be contributing to the burden of overweight/obesity. Further studies are warranted, including whether addressing NSES disparity could reduce the risk of overweight/obesity.PROSPERO registration numberCRD42017063889


2019 ◽  
Vol 4 (1) ◽  
Author(s):  
Pauline A. J. Steegmans ◽  
Nicola Di Girolamo ◽  
Reint A. Meursinge Reynders

Abstract Background Titles and abstracts are the most read sections of biomedical papers. It is therefore important that abstracts transparently report both the beneficial and adverse effects of health care interventions and do not mislead the reader. Misleading reporting, interpretation, or extrapolation of study results is called “spin”. In this study, we will assess whether adverse effects of orthodontic interventions were reported or considered in the abstracts of both Cochrane and non-Cochrane reviews and whether spin was identified and what type of spin. Methods Eligibility criteria were defined for the type of study designs, participants, interventions, outcomes, and settings. We will include systematic reviews of clinical orthodontic interventions published in the five leading orthodontic journals and in the Cochrane Database. Empty reviews will be excluded. We will manually search eligible reviews published between 1 August 2009 and 31 July 2019. Data collection forms were developed a priori. All study selection and data extraction procedures will be conducted by two reviewers independently. Our main outcomes will be the prevalence of reported or considered adverse effects of orthodontic interventions in the abstract of systematic reviews and the prevalence of “spin” related to these adverse effects. We will also record the prevalence of three subtypes of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation-related spin. All statistics will be calculated for the following groups: (1) all journals individually, (2) all journals together, and (3) the five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately. Generalized linear models will be developed to compare the various groups. Discussion We expect that our results will raise the awareness of the importance of reporting and considering of adverse effects and the presence of the phenomenon of spin related to these effects in abstracts of systematic reviews of orthodontic interventions. This is important, because an incomplete and inadequate reporting, interpretation, or extrapolation of findings on adverse effects in abstracts of systematic reviews can mislead readers and could lead to inadequate clinical practice. Our findings could result in policy implications for making judgments about the acceptance for publication of systematic reviews of orthodontic interventions.


2019 ◽  
Author(s):  
Claudia Hacke ◽  
David Nunan

AbstractObjectiveTo explore factors underpinning discrepancies in reported pooled effect estimates from Cochrane and non-Cochrane systematic reviews answering the same question.Study Design and SettingWe observed discrepant pooled effects in 23 out of 24 pairs of meta-analyses from Cochrane and non-Cochrane systematic reviews answering the same question. Here we present the results of a systematic assessment of methodological quality and factors that explain the observed quantitative discrepancies. Methodological quality of each review was assessed using AMSTAR (Assessing the Methodological Quality of Systematic Reviews). Matched pairs were contrasted at the macro- (review methodology), meso- (application of methodology) and micro- (data extraction) level and reasons for differences were derived.ResultsAll Cochrane reviews had high methodological quality (AMSTAR 8-11), whereas the majority (87.5%) of non-Cochrane reviews were classified as moderate (AMSTAR 4-7). Only one pair included exactly the same studies for their respective meta-analyses but there was still a discrepancy in the pooled estimate due to differences in data extraction. One pair did not include any study of its match and for one pair the same effect estimates were reported despite inclusion of different studies. The remaining pairs included at least one study in their match. Due to insufficient reporting (predominantly affecting non-Cochrane reviews) we were only able to completely ascertain the reasons for discrepancies in all included studies for 9/24 (37.5%) pairs. Across all pairs, differences in pre-defined methods (macro-level) including search strategy, eligibility criteria and performance of dual screening could possibly explain mismatches in included studies. Study selection procedures (meso-level) including disagreements in the interpretation of pre-defined eligibility criteria (14 matches) were identified as reasons underpinning discrepant review findings. Comparison of data extraction from primary studies (micro-level) was not possible in 13/24 pairs as a result of the non-Cochrane review providing insufficient details of the studies included in their meta-analyses. Two out of 24 pairs completely agreed on the numerical data presented for the same studies in their respective meta-analysis. Both review types provided sufficient information to check the accuracy of data extraction for 8 pairs (45 studies) where there were discrepancies. An assessment of 50% (22 studies) of these showed that reasons for differences in extracted data could be identified in 15 studies. We found examples for both types of review where data presented were discrepant from that given in the source study without a plausible explanation.ConclusionMethodological and author judgements and performance are key aspects underpinning poor overlap of included studies and discrepancies in reported pooled effect estimates between topic-matched reviews. Though caution must be taken when extrapolating, our findings raise the question as to what extent the entire meta-analysis evidence-base accurately reflects the available primary research both in terms of volume and data. Reinforcing awareness of the application of guidelines for systematic reviews and meta-analyses may help mitigate some of the key issues identified in our analysis.What is new?Key findings Non-Cochrane reviews were of a lower overall methodological quality compared with Cochrane reviews. Discrepant results of meta-analyses on the same topic can be attributed to differences in included studies based on review author decision, judgements and performance at different stages of the review process.What this adds to what was known?This study provides the most robust analysis to date of the potential methodological factors underpinning discrepant review findings between matched meta-analyses answering the same question. Assessing differences between reviews at the macro-, meso-, and micro-levels is a useful method to identify reasons for discrepant meta-analyses at key stages of the review process.What is the implication and what should change now?There is a need for a standardised approach to performing matched-pair analysis of meta-analyses and systematic reviews answering the same question. Our paper provides a base for this that can be refined by replication and expert consensus.


2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Lynn Hendricks ◽  
Ingrid Eshun-Wilson ◽  
Anke Rohwer

Abstract Background People living with human immunodeficiency virus (PLHIV) struggle with the challenges of living with a chronic disease and integrating antiretroviral treatment (ART) and care into their daily lives. The aims of this study were as follows: (1) to undertake the first mega-aggregation of qualitative evidence syntheses using the methods of framework synthesis and (2) make sense of existing qualitative evidence syntheses that explore the barriers and facilitators of adherence to antiretroviral treatment, linkage to care and retention in care for PLHIV to identify research gaps. Methods We conducted a comprehensive search and did all screening, data extraction and critical appraisal independently and in duplicate. We used the Kaufman HIV Behaviour Change model (Kaufman et al., 2014) as a framework to synthesise the findings using the mega-aggregative framework synthesis approach, which consists of 8 steps: (1) identify a clearly defined review question and objectives, (2) identify a theoretical framework or model, (3) decide on criteria for considering reviews for inclusion, (4) conduct searching and screening, (5) conduct quality appraisal of the included studies, (6) data extraction and categorisation, (7) present and synthesise the findings, and (8) transparent reporting. We evaluated systematic reviews up to July 2018 and assessed methodological quality, across reviews, using the Joanna Briggs Institute Critical Appraisal Checklist for Systematic Reviews. Results We included 33 systematic reviews from low, middle- and high-income countries, which reported on 1,111,964 PLHIV. The methodological quality of included reviews varied considerably. We identified 544 unique third-order concepts from the included systematic reviews, which were reclassified into 45 fourth-order themes within the individual, interpersonal, community, institutional and structural levels of the Kaufman HIV Behaviour Change model. We found that the main influencers of linkage, adherence and retention behaviours were psychosocial and personal characteristics—perceptions of ART, desires, fears, experiences of HIV and ART, coping strategies and mental health issues—interwoven with other factors on the interpersonal, community, institutional and structural level. Using this approach, we found interdependence between factors influencing ART linkage, retention and adherence and identified the need for qualitative evidence that explores, in greater depth, the complex relationships between structural factors and adherence, sociodemographic factors such as community violence and retention, and the experiences of growing up with HIV in low- and middle-income countries—specifically in children, youth, women and key populations. Conclusions This is the first mega-aggregation framework synthesis, or synthesis of qualitative evidence syntheses using the methods of framework synthesis at the overview level. We found the novel method to be a transparent and efficient method for assessing the quality and making sense of existing qualitative systematic reviews. Systematic review registration The protocol of this overview was registered on PROSPERO (CRD42017078155) on 17 December 2017.


BMJ Open ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. e041680
Author(s):  
Shu-Yue Pan ◽  
Rui-Juan Cheng ◽  
Zi-Jing Xia ◽  
Qiu-Ping Zhang ◽  
Yi Liu

ObjectivesGout, characterised by hyperuricaemia with monosodium urate crystal formation and inflammation, is the most common inflammatory arthritis in adults. Recent studies have found that elevated uric acid levels are related to the occurrence of dementia. We conducted a study to investigate the association between dementia and gout or hyperuricaemia.DesignSystematic review and meta-analysis of cohort studies.Data sourcesStudies were screened from inception to 28 June 2019 by searching Medline, Embase and the Cochrane Library databases.Eligibility criteriaCohort studies comparing the risk of dementia in patients with gout and hyperuricaemia versus non-gout and non-hyperuricaemia controls were enrolled.Data extraction and analysisTwo reviewers separately selected studies and extracted data using the Medical Subject Headings without restriction on languages or countries. The adjusted HRs were pooled using the DerSimonian and Laird random effects model. Sensitivity analyses were conducted to evaluate the stability of the results. Publication bias was evaluated using Egger’s and Begg’s tests. Quality assessment was performed according to the Newcastle-Ottawa Scale.ResultsFour cohort studies that met the inclusion criteria were included in our meta-analysis. We found that gout and hyperuricaemia did not increase the risk of dementia, with a pooled HR of 0.94 (95% CI 0.69 to 1.28), but might decrease the risk of Alzheimer’s disease (AD), with a pooled HR of 0.78 (95% CI 0.64 to 0.95). There was little evidence of publication bias. Quality assessment of the included studies was high (range: 6–8 points).ConclusionsOur study shows that gout and hyperuricaemia do not increase the risk of dementia. However, gout and hyperuricaemia might have a protective effect against AD. Due to the limited number of research articles, more investigations are needed to demonstrate the potential relationship between dementia and gout or hyperuricaemia.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Katie O’Hearn ◽  
Cameron MacDonald ◽  
Anne Tsampalieros ◽  
Leo Kadota ◽  
Ryan Sandarage ◽  
...  

Abstract Background Standard practice for conducting systematic reviews (SRs) is time consuming and involves the study team screening hundreds or thousands of citations. As the volume of medical literature grows, the citation set sizes and corresponding screening efforts increase. While larger team size and alternate screening methods have the potential to reduce workload and decrease SR completion times, it is unknown whether investigators adapt team size or methods in response to citation set sizes. Using a cross-sectional design, we sought to understand how citation set size impacts (1) the total number of authors or individuals contributing to screening and (2) screening methods. Methods MEDLINE was searched in April 2019 for SRs on any health topic. A total of 1880 unique publications were identified and sorted into five citation set size categories (after deduplication): < 1,000, 1,001–2,500, 2,501–5,000, 5,001–10,000, and > 10,000. A random sample of 259 SRs were selected (~ 50 per category) for data extraction and analysis. Results With the exception of the pairwise t test comparing the under 1000 and over 10,000 categories (median 5 vs. 6, p = 0.049) no statistically significant relationship was evident between author number and citation set size. While visual inspection was suggestive, statistical testing did not consistently identify a relationship between citation set size and number of screeners (title-abstract, full text) or data extractors. However, logistic regression identified investigators were significantly more likely to deviate from gold-standard screening methods (i.e. independent duplicate screening) with larger citation sets. For every doubling of citation size, the odds of using gold-standard screening decreased by 15 and 20% at title-abstract and full text review, respectively. Finally, few SRs reported using crowdsourcing (n = 2) or computer-assisted screening (n = 1). Conclusions Large citation set sizes present a challenge to SR teams, especially when faced with time-sensitive health policy questions. Our study suggests that with increasing citation set size, authors are less likely to adhere to gold-standard screening methods. It is possible that adjunct screening methods, such as crowdsourcing (large team) and computer-assisted technologies, may provide a viable solution for authors to complete their SRs in a timely manner.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Carole Lunny ◽  
Dawid Pieper ◽  
Pierre Thabet ◽  
Salmaan Kanji

Abstract Background Overviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap. Methods We first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use. Results Nine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview – the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview. Conclusions Our study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.


2017 ◽  
Vol 27 (6) ◽  
pp. 619-627 ◽  
Author(s):  
V. C. H. Chung ◽  
X. Y. Wu ◽  
Y. Feng ◽  
R. S. T. Ho ◽  
S. Y. S. Wong ◽  
...  

Aims.Depression is one of the most common mental disorders and identifying effective treatment strategies is crucial for the control of depression. Well-conducted systematic reviews (SRs) and meta-analyses can provide the best evidence for supporting treatment decision-making. Nevertheless, the trustworthiness of conclusions can be limited by lack of methodological rigour. This study aims to assess the methodological quality of a representative sample of SRs on depression treatments.Methods.A cross-sectional study on the bibliographical and methodological characteristics of SRs published on depression treatments trials was conducted. Two electronic databases (the Cochrane Database of Systematic Reviews and the Database of Abstracts of Reviews of Effects) were searched for potential SRs. SRs with at least one meta-analysis on the effects of depression treatments were considered eligible. The methodological quality of included SRs was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. The associations between bibliographical characteristics and scoring on AMSTAR items were analysed using logistic regression analysis.Results.A total of 358 SRs were included and appraised. Over half of included SRs (n = 195) focused on non-pharmacological treatments and harms were reported in 45.5% (n = 163) of all studies. Studies varied in methods and reporting practices: only 112 (31.3%) took the risk of bias among primary studies into account when formulating conclusions; 245 (68.4%) did not fully declare conflict of interests; 93 (26.0%) reported an ‘a priori’ design and 104 (29.1%) provided lists of both included and excluded studies. Results from regression analyses showed: more recent publications were more likely to report ‘a priori’ designs [adjusted odds ratio (AOR) 1.31, 95% confidence interval (CI) 1.09–1.57], to describe study characteristics fully (AOR 1.16, 95% CI 1.06–1.28), and to assess presence of publication bias (AOR 1.13, 95% CI 1.06–1.19), but were less likely to list both included and excluded studies (AOR 0.86, 95% CI 0.81–0.92). SRs published in journals with higher impact factor (AOR 1.14, 95% CI 1.04–1.25), completed by more review authors (AOR 1.12, 95% CI 1.01–1.24) and SRs on non-pharmacological treatments (AOR 1.62, 95% CI 1.01–2.59) were associated with better performance in publication bias assessment.Conclusion.The methodological quality of included SRs is disappointing. Future SRs should strive to improve rigour by considering of risk of bias when formulating conclusions, reporting conflict of interests and authors should explicitly describe harms. SR authors should also use appropriate methods to combine the results, prevent language and publication biases, and ensure timely updates.


BMJ Open ◽  
2021 ◽  
Vol 11 (7) ◽  
pp. e051224
Author(s):  
Vaidehi Misra ◽  
Frozan Safi ◽  
Kathryn A Brewerton ◽  
Wei Wu ◽  
Robin Mason ◽  
...  

ObjectivesEvaluate gender differences in authorship of COVID-19 articles in high-impact medical journals compared with other topics.DesignCross-sectional review.Data sourcesMedline database.Eligibility criteriaArticles published from 1 January to 31 December 2020 in the seven leading general medical journals by impact factor. Article types included primary research, reviews, editorials and commentaries.Data extractionKey data elements were whether the study topic was related to COVID-19 and names of the principal and the senior authors. A hierarchical approach was used to determine the likely gender of authors. Logistic regression assessed the association of study characteristics, including COVID-19 status, with authors’ likely gender; this was quantified using adjusted ORs (aORs).ResultsWe included 2252 articles, of which 748 (33.2%) were COVID-19-related and 1504 (66.8%) covered other topics. A likely gender was determined for 2138 (94.9%) principal authors and 1890 (83.9%) senior authors. Men were significantly more likely to be both principal (1364 men; 63.8%) and senior (1332 men; 70.5%) authors. COVID-19-related articles were not associated with the odds of men being principal (aOR 0.99; 95% CI 0.81 to 1.21; p=0.89) or senior authors (aOR 0.96; 95% CI 0.78 to 1.19; p=0.71) relative to other topics. Articles with men as senior authors were more likely to have men as principal authors (aOR 1.49; 95% CI 1.21 to 1.83; p<0.001). Men were more likely to author articles reporting original research and those with corresponding authors based outside the USA and Europe.ConclusionsWomen were substantially under-represented as authors among articles in leading medical journals; this was not significantly different for COVID-19-related articles. Study limitations include potential for misclassification bias due to the name-based analysis. Results suggest that barriers to women’s authorship in high-impact journals during COVID-19 are not significantly larger than barriers that preceded the pandemic and that are likely to continue beyond it.PROSPERO registration numberCRD42020186702.


2006 ◽  
Vol 1 (4) ◽  
pp. 3 ◽  
Author(s):  
Li Zhang ◽  
Margaret Sampson ◽  
Jessie McGowan

Introduction - This study applied the principles of evidence based information practice to clarify the role of information specialists and librarians in the preparation of Cochrane systematic reviews and to determine whether information specialists impact the quality of searching in Cochrane systematic reviews. Objectives - This research project sought to determine how the contribution of the person responsible for searching in the preparation of Cochrane systematic reviews was reported; whether the contribution was recognized through authorship or acknowledgement; the qualifications of the searcher; and the association between the type of contributorship and characteristics of the search strategy, assessability, and the presence of certain types of errors. Methods - Data sources: The Cochrane Database of Systematic Reviews, The Cochrane Library 3 (2002). Inclusion criteria: The study included systematic reviews that met the following criteria: one or more sections of the Cochrane Highly Sensitive Search Strategy were utilised, primary studies were either randomised controlled trials (RCTs) or quasi-RCTs, and included and excluded studies were clearly identified. Data extraction: Two librarians assessed the searches for errors, establishing consensus on discordant ratings. Results - Of the 169 reviews screened for this project, 105 met all eligibility criteria. Authors fulfilled the searching role in 41.9% of reviews studied, acknowledged persons or groups in 13.3%, a combination in 9.5%, and the role was not reported in 35.2% of reviews. For the 78 reviews in which meta-analyses were performed, the positions of those responsible for statistical decisions were examined for comparative purposes. The statistical role was performed by an author in 47.4% of cases and unreported in the same number of cases. Insufficient analyzable data was obtained regarding professional qualifications (3/105 for searching, 2/78 for statistical decisions). Search quality was assessed for 66 searches across 74 reviews. In general, it was more possible to assess the search quality when the searcher role was reported. An association was found between the reporting of searcher role and the presence of a consequential error. There was no association between the number of consequential errors and how the contribution of the searcher was reported. Conclusions - Qualifications of the persons responsible for searching and statistical decision-making were poorly reported in Cochrane reviews, but more complete role reporting is associated with greater assessability of searches and fewer substantive errors in search strategies.


Sign in / Sign up

Export Citation Format

Share Document