Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

Background In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs). Methods Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity). Results The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = − 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer’s unit, suggesting face validity. Conclusions Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach. Trial registration Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844.

Dignity therapy intervention fidelity: a cross-sectional descriptive study with older adult outpatients with cancer

Objectives Intervention fidelity is imperative to ensure confidence in study results and intervention replication in research and clinical settings. Like many brief protocol psychotherapies, Dignity Therapy lacks sufficient evidence of intervention fidelity. To overcome this gap, our study purpose was to examine intervention fidelity among therapists trained with a systematized training protocol. Methods For preliminary fidelity evaluation in a large multi-site stepped wedge randomized controlled trial, we analyzed 46 early transcripts of interviews from 10 therapists (7 female; 7 White, 3 Black). Each transcript was evaluated with the Revised Dignity Therapy Adherence Checklist for consistency with the Dignity Therapy protocol in terms of its Process (15 dichotomous items) and Core Principles (6 Likert-type items). A second rater independently coded 26% of the transcripts to assess interrater reliability. Results Each therapist conducted 2 to 10 interviews. For the 46 scored transcripts, the mean Process score was 12.4/15 (SD = 1.2), and the mean Core Principles score was 9.9/12 (SD = 1.8) with 70% of the transcripts at or above the 80% fidelity criterion. Interrater reliability (Cohen’s kappa and weighted kappa) for all Adherence Checklist items ranged between .75 and 1.0. For the Core Principles items, Cronbach’s alpha was .92. Conclusions Preliminary findings indicate that fidelity to Dignity Therapy delivery was acceptable for most transcripts and provide insights for improving consistency of intervention delivery. The systematized training protocol and ongoing monitoring with the fidelity audit tool will facilitate consistent intervention delivery and add to the literature about fidelity monitoring for brief protocol psychotherapeutic interventions.

Mixed-methods Process Evaluation of a Residence-based SARS-CoV-2 Testing Participation Pilot on a UK University Campus during the COVID-19 Pandemic

BackgroundRegular testing for SARS Coronavirus-2 (SARS-CoV-2) is an important strategy for controlling virus outbreaks on university campuses during the COVID-19 pandemic but testing participation rates can be low. The Residence-Based Testing Participation Pilot (RB-TPP) was a novel intervention implemented at two student residences on a large UK university campus over four weeks. The aim of the pilot was to increase the frequency of asymptomatic SARS-CoV-2 saliva testing onsite. This process evaluation aimed to determine whether RB-TPP was implemented as planned and identify implementation barriers and facilitators.MethodsA mixed-methods process evaluation was conducted alongside the RB-TPP. Evaluation participants were students (opting in, or out of RB-TPP) and staff with a role in service provision or student support. Monitoring data were collected from the intervention delivery team and meeting records. Data were collected from students via online survey (n=152) and seven focus groups (n=30), and from staff via individual interviews (n=13). Quantitative data were analysed descriptively and qualitative data thematically. Barriers and facilitators to implementation were mapped to the ‘Capability, Opportunity, Motivation–Behaviour’ (COM-B) behaviour change framework.Results464 students opted to participate in RB-TPP (98% of students living onsite). RB-TPP was implemented broadly as planned but was terminated early due to concerns relating to national escalation of the COVID-19 Delta variant. Most students (97.9%) perceived the relaxing of social distancing regulations within residences positively. The majority engaged in asymptomatic testing (88%); 46% (52% of testers) were fully compliant with pre-determined testing frequency. Implementation was facilitated by convenience and efficiency of testing, reduction in the risk of self-isolation and opportunities for students to socialise. Main barriers to implementation were mixed-messages about the rules, ambivalent attitudes, and lack of adherence to COVID-19 protective measures in the minority.ConclusionsThis process evaluation identifies factors that help or hinder the success of university residence-based outbreak prevention and management strategies. RB-TPP led to increased rates of COVID-19 testing participation among students in university residences. Perceived normalisation of university life significantly enhanced student mental wellbeing. The complexity and challenge generated by multiple lines of communication and rapid adaptions to a changing pandemic context was evident.Trial registration numberUKAS 307727-02-01; Pre-results. ClinicalTrials.gov Identifier: NCT05045989; post-results (first posted, 16/09/21).Ethical ApprovalFMHS 96-0920

The Effectiveness of Exercise Interventions Supported by Telerehabilitation For Recently Hospitalized Adult Medical Patients: A Systematic Review

Objective: To evaluate the effectiveness of exercise interventions delivered via telerehabilitation (via videoconference) for recently hospitalized adult medical patients. Data sources: A search was undertaken across six databases for English language publications from inception to May 2020. Methods: Studies were selected if they included an exercise intervention for recently hospitalized adults, delivered by a physiotherapist via videoconference. Two reviewers independently screened 1,122 articles (21 full text screening) and assessed methodological quality using the Downs and Black Checklist. A narrative synthesis of the included studies was undertaken. Results: Three studies met eligibility criteria involving 201 participants with chronic heart failure or chronic obstructive pulmonary disease. Findings demonstrated limited evidence supporting the effectiveness of exercise delivered via telerehabilitation in improving physical function and patient reported quality of life outcomes in recently hospitalized medical patients. Telerehabilitation in this setting was also associated with high attendance rates and patient satisfaction. Conclusions: Findings provide preliminary support for the benefits of exercise interventions delivered via telerehabilitation for recently hospitalized medical patients. Results do need to be interpreted with caution as further high-quality studies specific to this method of exercise intervention delivery are needed.

Cell phone availability and usage for mobile health and intervention delivery to persons living with HIV in a low-resource setting. (Preprint)

BACKGROUND HIV/AIDS is now a chronic illness due to effective antiretroviral therapy (ART) and involves routine follow-up care, including regular physical visits to the clinic. In the recent past, and in wake of the COVID-19 pandemic, there has been increased need for virtual care and intervention delivery, a modality known as mHealth, which includes cell phone-delivered services OBJECTIVE Here we describe cell phone use and its relationship with alcohol use in a cohort of persons with HIV (PWH) and latent tuberculosis (TB). METHODS We performed a cross-sectional analysis of baseline data from a cohort of PWH and latent TB in HIV care in south-western Uganda. We estimated proportions of cell phone and text message use and evaluated their associations with alcohol use, a common modifiable behaviour among PWH. Cell phone use (primary outcome) was defined as owning a cell phone that is turned on at least half of the day. Any alcohol use was defined as any self-reported alcohol use in the prior 3 months, and/or phosphatidylethanol (an alcohol biomarker) ≥8 ng/ml. RESULTS A total of 300 participants were included in the analysis, with a median age of 40 years; 48.7% were male. Majority (89.0%) of participants had access to a phone and of these 9.7% shared the phone with someone else. Eighty-seven percent owned a cell phone that is turned on at least half of the time; the majority (89.7%) rarely or never sent text messages, and over two-thirds (66.9%) rarely or never received text messages. 71.3% had any alcohol use. In adjusted analyses, any alcohol use was not significantly associated with cell phone use (adjusted OR [aOR]: 0.48, 95% confidence interval [CI]: 0.18-1.25, p-value=.13), or sending (aOR: 0.82, 95% CI: 0.28-2.37, p-value=.71) or receiving (aOR: 1.31, 95% CI: 0.70-2.47, p-value= .40) of text messages. CONCLUSIONS There is hope that mHealth interventions in this population can be carried out using cell phones due to their popularity; however, the interventions may need to employ methods that do not rely on the sending and receiving of text messages only. CLINICALTRIAL NCT 03302299

The DIAMONDS intervention to support self-management of type 2 diabetes in people with severe mental illness: study protocol for a single-group feasibility study

IntroductionThe DIAMONDS programme aims to evaluate a novel supported diabetes self-management intervention for people with severe mental illness (the “DIAMONDS intervention”). The purpose of this study is to test the feasibility of intervention delivery and data collection procedures to inform a definitive randomised controlled trial (RCT).MethodsAdults aged 18 years or over with a diagnosis of type 2 diabetes and severe mental illness (schizophrenia, schizoaffective disorder, or bipolar disorder) will be eligible for inclusion. Individuals with other types of diabetes or non-psychotic mental illness and those lacking capacity to consent will not be eligible. Participants will be recruited from NHS mental health trusts and general practices across the North of England. All participants will receive the DIAMONDS intervention: weekly one-to-one sessions with a trained facilitator (“DIAMONDS Coach”) to support goal setting, action planning, and diabetes education; ongoing self-management supported by a paper-based workbook and optional digital application (app); and monthly peer-support group sessions with other participants. The primary outcomes are: 1. Recruitment rate, measured as proportion of the recruitment target (N=30) achieved at 5 months from start of recruitment, 2. Attrition measured as the proportion of missing outcomes data at the end of the recruitment period (5 months from start of recruitment) for physiological and self-reported data items, 3. Intervention delivery rate recorded as the proportion of planned sessions delivered (measured by the number of completed intervention session logs per participant within 15 weeks of the first intervention session). Secondary outcomes include completeness of data collection at baseline and of process evaluation data at follow-up as well as the feasibility and acceptability of the intervention and of wearing a blinded continuous glucose monitoring device. An intervention fidelity framework will also be developed. Recruitment started in July 2021. The study was prospectively registered: ISRCTN15328700 (12th March 2021).DiscussionThe results of this feasibility study will inform the refinement of the content and delivery of the DIAMONDS intervention, as well as research procedures, including recruitment and data collection, in preparation for the main DIAMONDS RCT.

Clinicians for CARE: A Systematic Review and Meta‐Analysis of Interventions to Support Caregivers of Patients With Heart Disease

Background Caregivers provide critical support for patients with chronic diseases, including heart disease, but often experience caregiver stress that negatively impacts their health, quality of life, and patient outcomes. We aimed to inform health care teams on an evidence‐based approach to supporting the caregivers of patients with heart disease. Methods and Results We conducted a systematic review and meta‐analysis of randomized controlled trials written in English that evaluated interventions to support caregivers of patients with heart disease. We identified 15,561 articles as of April 2, 2020 from 6 databases; of which 20 unique randomized controlled trials were evaluated, representing a total of 1570 patients and 1776 caregivers. Most interventions focused on improving quality of life, and reducing burden, depression, and anxiety; 85% (17 of 20) of the randomized controlled trials provided psychoeducation for caregivers. Interventions had mixed results, with moderate non‐significant effects observed for depression (Hedges’ g=−0.64; 95% CI, −1.34 to 0.06) and burden (Hedges’ g=−0.51; 95% CI, −2.71 to 1.70) at 2 to 4 months postintervention and small non‐significant effects observed for quality of life and anxiety. These results were limited by the heterogeneity of outcome measures and intervention delivery methods. A qualitative synthesis of major themes of the interventions resulted in clinical recommendations represented with the acronym “CARE” (Caregiver‐Centered, Active engagement, Reinforcement, Education). Conclusions This systematic review highlights the need for greater understanding of the challenges faced by caregivers and the development of guidelines to help clinicians address those challenges. More research is necessary to develop clinical interventions that consistently improve caregiver outcomes.

Implementation of Video Feedback Within a Community Based Naturalistic Developmental Behavioral Intervention Program for Toddlers With ASD: Pilot Study

Video feedback (VF) is an intervention delivery technique that complements naturalistic developmental behavioral interventions (NDBI) and parent-mediated interventions (PMI) by using caregiver-child interaction videos reviewed with a clinician to facilitate behavioral change in caregivers. Although VF has been implemented in PMI with young children with ASD, examinations of feasibility and acceptability, as well as the potential effectiveness of VF in community settings, have been limited. In this pilot randomized control trial (NCT03397719; https://clinicaltrials.gov/ct2/show/NCT03397719), families were randomized into a state-funded Early Intervention (EI) NDBI program or the NDBI program augmented with VF. Results demonstrated high levels of implementation and acceptability of VF augmenting the community-based EI program in caregivers and clinicians. Both groups showed significant improvements after 6 months in social communication symptoms and some areas of developmental and adaptive skills.Clinical Trial Registration:https://clinicaltrials.gov/ct2/show/NCT03397719, identifier: NCT03397719.

Maintaining Integrity of Worldwide FINGER Clinical Trials During the COVID-19 Pandemic

The COVID-19 pandemic presents challenges to the conduct of randomized clinical trials of lifestyle interventions. World-Wide FINGERS international network convened a forum for researchers to discuss statistical design and analysis issues they faced during the pandemic. We report experiences of three trials that, at various stages of conduct, altered designs and analysis plans to navigate these issues. We provide recommendations for future trials to consider as they develop and launch behavioral intervention trials. The pandemic led researchers to change recruitment plans, interrupt timelines for assessments and intervention delivery, and move to remote intervention and assessments protocols. The necessity of these changes add emphasis to the importance, in study design and analysis, of intention to treat approaches, flexibility, within site stratification, interim power projections, and sensitivity analyses. Robust approaches to study design and analysis are critical to negotiate issues related to the intervention.