scholarly journals Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

Trials ◽  
2022 ◽  
Vol 23 (1) ◽  
Author(s):  
Miranda B. Olson ◽  
Ellen M. McCreedy ◽  
Rosa R. Baier ◽  
Renée R. Shield ◽  
Esme E. Zediker ◽  
...  

Abstract Background In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs). Methods Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity). Results The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = − 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer’s unit, suggesting face validity. Conclusions Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach. Trial registration Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844.

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Machelle Wilchesky ◽  
Stephanie A. Ballard ◽  
Philippe Voyer ◽  
Jane McCusker ◽  
Ovidiu Lungu ◽  
...  

Abstract Background Delirium is a significant cause of morbidity and mortality among older people admitted to both acute and long-term care facilities (LTCFs). Multicomponent interventions have been shown to reduce delirium incidence in the acute care setting (30–73%) by acting on modifiable risk factors. Little work, however, has focused on using this approach to reduce delirium incidence in LTCFs. Methods The objective is to assess the effectiveness of the multicomponent PREPARED Trial intervention in reducing the following primary outcomes: incidence, severity, duration, and frequency of delirium episodes in cognitively impaired residents. This 4-year, parallel-design, cluster randomized study will involve nursing staff and residents in 45–50 LTCFs in Montreal, Canada. Participating public and private LTCFs (clusters) that provide 24-h nursing care will be assigned to either the PREPARED Trial intervention or the control (usual care) arm of the study using a covariate constrained randomization procedure. Approximately 400–600 LTC residents aged 65 and older with dementia and/or cognitive impairment will be enrolled in the study and followed for 18 weeks. Residents must be at risk of delirium, delirium-free at baseline and have resided at the facility for at least 2 weeks. Residents who are unable to communicate verbally, have a history of specific psychiatric conditions, or are receiving end-of-life care will be excluded. The PREPARED Trial intervention consists of four main components: a decision tree, an instruction manual, a training package, and a toolkit. Primary study outcomes will be assessed weekly. Functional autonomy and cognitive levels will be assessed at the beginning and end of follow-up, while information pertaining to modifiable delirium risk factors, medical consultations, and facility transfers will be collected retrospectively for the duration of the follow-up period. Primary outcomes will be reported at the level of intervention assignment. All researchers analyzing the data will be blinded to group allocation. Discussion This large-scale intervention study will contribute significantly to the development of evidence-based clinical guidelines for delirium prevention in this frail elderly population, as it will be the first to evaluate the efficacy of a multicomponent delirium prevention program translated into LTC clinical practice on a large scale. Trial registration NCT03718156, ClinicalTrials.gov.


Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Allyson L. Dir ◽  
Dennis P. Watson ◽  
Matthew Zhiss ◽  
Lisa Taylor ◽  
Bethany C. Bray ◽  
...  

Abstract Background This manuscript provides a research update to the ongoing pragmatic trial of Project POINT (Planned Outreach, Intervention, Naloxone, and Treatment), an emergency department-based peer recovery coaching intervention for linking patients with opioid use disorder to evidence-based treatment. The research team has encountered a number of challenges related to the “real-world” study setting since the trial began. Using an implementation science lens, we sought to identify and describe barriers impacting both the intervention and research protocols of the POINT study, which are often intertwined in pragmatic trials due to the focus on external validity. Method Qualitative data were collected from 3 peer recovery coaches, 2 peer recovery coach supervisors, and 3 members of the research team. Questions and deductive qualitative analysis were guided by the Consolidated Framework for Implementation Research (CFIR). Results Nine unique barriers were noted, with 5 of these barriers impacting intervention and research protocol implementation simultaneously. These simultaneous barriers were timing of intervention delivery, ineffective communication with emergency department staff, lack of privacy in the emergency department, the fast-paced emergency department setting, and patient’s limited resources. Together, these barriers represent the intervention characteristics, inner setting, and outer setting domains of the CFIR. Conclusion Results highlight the utility of employing an implementation science framework to assess implementation issues in pragmatic trials and how this approach might be used as a quality assurance mechanism given the considerable overlap that exists between research and intervention protocols in real-world trial settings. Previously undocumented changes to the trial design that have been made as a result of the identified barriers are discussed.


Author(s):  
Eun-Hi Kong ◽  
Myoungsuk Kim ◽  
Seonho Kim

Physical restraint is still frequently used in many countries. However, a lack of education hinders physical restraint reduction in long-term care facilities. No study has yet to examine the effects of physical restraint reduction education on nursing students. This study aimed to evaluate the effects of a web-based educational program of physical restraint reduction on nursing students’ knowledge and perceptions. A cluster randomized controlled and single-blind design was used. This study was conducted at four nursing schools in South Korea. A total of 169 undergraduate nursing students completed this study. Using random allocation, two nursing schools (85 students) were allocated as the experimental group and the other two schools (84 students) as the control group. The experimental group received the web-based educational program, and the control group did not receive the educational program. Data were collected immediately before and after the web-based educational program. The experimental group’s knowledge and perceptions significantly improved between pre-test and post-test. The analysis of covariance showed statistically significant differences between groups in knowledge (p < 0.001) and perceptions (p < 0.001) over time, revealing positive effects of the web-based educational program. The web-based educational program regarding physical restraint reduction positively affected nursing students’ knowledge and perceptions. Future studies are required to examine the educational program’s longitudinal effects with more rigorous measurements and research methods.


Author(s):  
Michele Heisler ◽  
Jennifer Burgess ◽  
Jeffrey Cass ◽  
John F. Chardos ◽  
Alexander B. Guirguis ◽  
...  

Abstract Objective To examine whether diabetes shared medical appointments (SMAs) implemented as part of usual clinical practice in diverse health systems are more effective than usual care in improving and sustaining A1c improvements. Research Design and Methods A multi-site cluster randomized pragmatic trial examining implementation in clinical practice of diabetes SMAs in five Veterans Affairs (VA) health systems was conducted from 2016 to 2020 among 1537 adults with type 2 diabetes and elevated A1cs. Eligible patients were randomly assigned to either: (1) invitation to participate in a series of SMAs totaling 8–9 h; or (2) continuation of usual care. Relative change in A1c (primary outcome) and in systolic blood pressure, insulin starts, statin starts, and anti-hypertensive medication classes (secondary outcomes) were measured as part of usual clinical care at baseline, at 6 months and at 12 months (~7 months after conclusion of the final SMA in four of five sites). We examined outcomes in three samples of SMA participants: all those scheduled for a SMA, those attending at least one SMA, and those attending at least half of SMAs. Results Baseline mean A1c was 9.0%. Participants scheduled for an SMA achieved A1c reductions 0.35% points greater than the control group between baseline and 6-months follow up (p = .001). Those who attended at least one SMA achieved reductions 0.42 % points greater (p < .001), and those who attended at least half of scheduled SMAs achieved reductions 0.53 % points greater (p < .001) than the control group. At 12-month follow-up, the three SMA analysis samples achieved reductions from baseline ranging from 0.16 % points (p = 0.12) to 0.29 % points (p = .06) greater than the control group. Conclusions Diabetes SMAs as implemented in real-life diverse clinical practices improve glycemic control more than usual care immediately after the SMAs, but relative gains are not maintained. Our findings suggest the need for further study of whether a longer term SMA model or other follow-up strategies would sustain relative clinical improvements associated with this intervention. Trial Registration ClinicalTrials.gov ID NCT02132676


BMC Nursing ◽  
2021 ◽  
Vol 20 (1) ◽  
Author(s):  
Olga L. Cortés ◽  
Mauricio Herrera-Galindo ◽  
Juan Carlos Villar ◽  
Yudi A. Rojas ◽  
María del Pilar Paipa ◽  
...  

Abstract Background Despite being considered preventable, ulcers due to pressure affect between 30 and 50% of patients at high and very high risk and susceptibility, especially those hospitalized under critical care. Despite a lack of evidence over the efficacy in prevention against ulcers due to pressure, hourly repositioning in critical care as an intervention is used with more or less frequency to alleviate pressure on patients’ tissues. This brings up the objective of our study, which is to evaluate the efficacy in prevention of ulcers due to pressure acquired during hospitalization, specifically regarding two frequency levels of repositioning or manual posture switching in adults hospitalized in different intensive care units in different Colombian hospitals. Methods A nurse-applied cluster randomized controlled trial of parallel groups (two branches), in which 22 eligible ICUs (each consisting of 150 patients), will be randomized to a high-frequency level repositioning intervention or to a conventional care (control group). Patients will be followed until their exit from each cluster. The primary result of this study is originated by regarding pressure ulcers using clusters (number of first ulcers per patient, at the early stage of progression, first one acquired after admission for 1000 days). The secondary results include evaluating the risk index on the patients’ level (Hazard ratio, 95% IC) and a description of repositioning complications. Two interim analyses will be performed through the course of this study. A statistical difference between the groups < 0.05 in the main outcome, the progression of ulcers due to pressure (best or worst outcome in the experimental group), will determine whether the study should be put to a halt/determine the termination of the study. Conclusion This study is innovative in its use of clusters to advance knowledge of the impact of repositioning as a prevention strategy against the appearance of ulcers caused by pressure in critical care patients. The resulting recommendations of this study can be used for future clinical practice guidelines in prevention and safety for patients at risk. Trial registration PENFUP phase-2 was Registered in Clinicaltrials.gov (NCT04604665) in October 2020.


BMJ Open ◽  
2019 ◽  
Vol 9 (8) ◽  
pp. e029565
Author(s):  
Barbara Seebacher ◽  
Roger J Mills ◽  
Markus Reindl ◽  
Laura Zamarian ◽  
Raija Kuisma ◽  
...  

IntroductionSelf-efficacy refers to individuals’ confidence in their ability to perform relevant tasks to accomplish desired goals. This is independent of their actual abilities. In people with multiple sclerosis (MS), self-efficacy has been shown to powerfully influence motivation and health-related behaviour, such as adherence to prescribed treatment or physical activity. So far, a rigorously tested German language self-efficacy questionnaire for people with MS is missing.MethodsThe purpose of this study is to translate the original Unidimensional Self-Efficacy Scale for Multiple Sclerosis (USE-MS) into German and to validate the German USE-MS (USE-MS-G). Based on Bandura’s concept of self-efficacy and international guidelines for questionnaire development, the patient-led development of the pre-final German version will involve a forward–backward translation process, synthesis of translations, expert committee review and consensus with the original test developers. At two centres in Tyrol, Austria, content and face validity and cultural adaption for Austria will be established using face-to-face semistructured cognitive interviews of 30 people with MS (PwMS). A further 292 PwMS with minimal to severe disability will be tested at two timepoints to validate the USE-MS-G.ResultsMixed methods analyses will be applied. Interviews will be transcribed and analysed employing qualitative content analysis. External validity will be explored using Spearman’s Rank correlation coefficients of the USE-MS-G with the 13-item Resilience Scale, General Self-Efficacy Scale, Multiple Sclerosis International Quality of Life questionnaire, Hospital Anxiety and Depression Scale and MS-specific Neurological Fatigue Index. Test–retest reliability, internal consistency and floor and ceiling effects will be evaluated. Internal validity will be examined using Rasch analysis.Ethics and disseminationEthical approval was received from the Ethics Committee of the Medical University of Innsbruck, Austria (reference number EK1260/2018; 13.12.2018). Results from this study will be disseminated to the participants and MS Societies, and to clinicians and researchers through peer-reviewed publications and conferences.Study registrationISRCTN Registry; trial ID ISRCTN14843579; prospectively registered on 02. 01. 2019; http://www.isrctn.com/ISRCTN14843579


2007 ◽  
Vol 191 (S50) ◽  
pp. s78-s84 ◽  
Author(s):  
Richard Hodgson ◽  
Chris Bushe ◽  
Robert Hunter

BackgroundRandomised controlled trials (RCTs) are the gold standard for evaluating treatment efficacy. However, the outcomes of RCTs often lackclinical utility and usually do not address real-world effectivenessAimsTo review how traditional RCTs may be triangulatedwith other methodologies such as observational studies and pragmatic trials by highlighting recently reported studies, outcomes used and their respective meritsMethodLiterature review focusing on drug treatmentResultsRecently reported observational and some pragmatic studies show a degree of consistency in reported results and use outcomes that have face validity for cliniciansConclusionsNo single experimental paradigm or outcome provides the necessary data to optimise treatment of mental illness in the clinical setting


Trials ◽  
2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Marc-Eric Nadeau ◽  
Justine L. Henry ◽  
Todd C. Lee ◽  
Émilie Bortolussi-Courval ◽  
Carole Goodine ◽  
...  

Abstract Background Medication overload or problematic polypharmacy is a major problem causing widespread harm, particularly to older adults. Taking multiple medications increases the risk of potentially inappropriate medications (PIMs), and residents in long-term care (LTC) are frequently prescribed 10 or more medications at once. One strategy to address this problem is for the physician and/or pharmacist to perform regular medication reviews; however, this process can be complicated and time-consuming. With a prescription review, medications may be decreased, changed, or stopped altogether. MedReviewRx is a software that runs an analysis using deprescribing rules to produce a report to guide medication reviews addressing medication overload for residents in LTC. Methods This study will employ a mixed methods effectiveness-implementation hybrid type 2 study design. To measure effectiveness, a stepped wedge cluster randomized trial design is planned, which allows us to approximate a randomized clinical trial. Approximately 1000 residents living in LTC will be recruited from five facilities in New Brunswick. The study will begin with 3 months of baseline data on rates of deprescribing. Thereafter, every 3 months a new cluster will enter the intervention mode. The intervention consists of medication reviews augmented with the MedReviewRx software, which will be used by staff and clinicians in the facilities. The estimated study duration is 18 months and the main outcome will be the proportion of patients with one or more PIMs deprescribed (reduced/stopped or changed to a safer alternative) in the 90 days following a prescription review. The goal is to study the impact of MedReviewRx on medication overload among older adults living in LTC. In typical fashion of a stepped wedge cluster randomized trial, each cluster acts as an internal control (before and after) as well as a control for the other clusters (external control). Qualitative data collected will include resident/caregiver attitudes towards deprescribing and semi-structured interviews with staff working in the long-term care homes. Discussion This study design addresses issues with seasonality and allows all clusters to participate in the intervention, which is an advantage when the intervention is related to quality improvement. This study will provide valuable information on PIM use, cost savings, and facilitators and challenges associated with medication reviews and deprescribing. This study represents an important step towards understanding and promoting tools to guide safe and rational reduction of PIM use among older adults. Trial registration NCT04762303, Registered February 21, 2021.


2020 ◽  
Vol 17 (3) ◽  
pp. 253-263 ◽  
Author(s):  
Monica Taljaard ◽  
Cory E Goldstein ◽  
Bruno Giraudeau ◽  
Stuart G Nicholls ◽  
Kelly Carroll ◽  
...  

Background: Novel rationales for randomizing clusters rather than individuals appear to be emerging from the push for more pragmatic trials, for example, to facilitate trial recruitment, reduce the costs of research, and improve external validity. Such rationales may be driven by a mistaken perception that choosing cluster randomization lessens the need for informed consent. We reviewed a random sample of published cluster randomized trials involving only individual-level health care interventions to determine (a) the prevalence of reporting a rationale for the choice of cluster randomization; (b) the types of explicit, or if absent, apparent rationales for the use of cluster randomization; (c) the prevalence of reporting patient informed consent for study interventions; and (d) the types of justifications provided for waivers of consent. We considered cluster randomized trials for evaluating exclusively the individual-level health care interventions to focus on clinical trials where individual randomization is only theoretically possible and where there is a general expectation of informed consent. Methods: A random sample of 40 cluster randomized trials were identified by implementing a validated electronic search filter in two electronic databases (Ovid MEDLINE and Embase), with two reviewers independently extracting information from each trial. Inclusion criteria were the following: primary report of a cluster randomized trial, evaluating exclusively an individual-level health care intervention, published between 2007 and 2016, and conducted in Canada, the United States, European Union, Australia, or low- and middle-income country settings. Results: Twenty-five trials (62.5%, 95% confidence interval = 47.5%–77.5%) reported an explicit rationale for the use of cluster randomization. The most commonly reported rationales were those with logistical or administrative convenience (15 trials, 60%) and those that need to avoid contamination (13 trials, 52%); five trials (20%) were cited rationales related to the push for more pragmatic trials. Twenty-one trials (52.5%, 95% confidence interval = 37%–68%) reported written informed consent for the intervention, two (5%) reported verbal consent, and eight (20%) reported waivers of consent, while in nine trials (22.5%) consent was unclear or not mentioned. Reported justifications for waivers of consent included that study interventions were already used in clinical practice, patients were not randomized individually, and the need to facilitate the pragmatic nature of the trial. Only one trial reported an explicit and appropriate justification for waiver of consent based on minimum criteria in international research ethics guidelines, namely, infeasibility and minimal risk. Conclusion: Rationales for adopting cluster over individual randomization and for adopting consent waivers are emerging, related to the need to facilitate pragmatic trials. Greater attention to clear reporting of study design rationales, informed consent procedures, as well as justification for waivers is needed to ensure that such trials meet appropriate ethical standards.


2019 ◽  
Vol 107 (2) ◽  
Author(s):  
Marianne Burke ◽  
Benjamin Littenberg

Objective: Providers’ use of clinical evidence technologies (CETs) improves their diagnosis and treatment decisions. Despite these benefits, few studies have evaluated the impact of CETs on patient outcomes. The investigators evaluated the effect of one CET, VisualDx, on skin problem outcomes in primary care.Methods: A cluster-randomized controlled pragmatic trial was conducted in outpatient clinics at an academic medical center in the northeastern United States. Participants were primary care providers (PCPs), and their adult patients were seen for skin problems. The intervention was VisualDx, as used by PCPs. Outcomes were patient-reported time from index clinic visit to problem resolution, and the number of follow-up visits to any provider for the same problem. PCPs who were randomly assigned to the intervention agreed to use VisualDx as their primary evidence source for skin problems. Control group PCPs agreed not to use VisualDx. Investigators collected outcome data from patients by phone at thirty-day intervals. Cox proportional hazards models assessed time to resolution. Wilcoxon-rank sum tests and logistic regression compared the need for return appointments.Results: Thirty-two PCPs and 433 patients participated. In proportional hazards modelling adjusted for provider clusters, the time from index visit to skin problem resolution was similar in both groups (hazard ratio=0.92; 95% confidence interval [CI]=0.70, 1.21; p=0.54). Patient follow-up appointments did not differ significantly between groups (odds ratio=1.26; CI=0.94, 1.70; p=0.29).Conclusion: This pragmatic trial tested the effectiveness of VisualDx on patient-reported skin disease outcomes in a generalizable clinical setting. There was no difference in skin problem resolution or number of follow-up visits when PCPs used VisualDx.


Sign in / Sign up

Export Citation Format

Share Document