Comparison of performance-based assessment and real world skill in people with serious mental illness: Ecological validity of the Test of Grocery Shopping Skills

2018 ◽  
Vol 266 ◽  
pp. 11-17 ◽  
Author(s):  
Laura A. Faith ◽  
Melisa V. Rempfer
2020 ◽  
Vol 1 ◽  
pp. 263348952094320
Author(s):  
Kelly A Aschbrenner ◽  
Gary R Bond ◽  
Sarah I Pratt ◽  
Kenneth Jue ◽  
Gail Williams ◽  
...  

Background: Limited empirical evidence exists on the impact of adaptations that occur in implementing evidence-based practices (EBPs) in real-world practice settings. The purpose of this study was to measure and evaluate adaptations to an EBP (InSHAPE) for obesity in persons with serious mental illness in a national implementation in mental health care settings. Methods: We conducted telephone interviews with InSHAPE provider teams at 37 (95%) of 39 study sites during 24-month follow-up of a cluster randomized trial of implementation strategies for InSHAPE at behavioral health organizations. Our team rated adaptations as fidelity-consistent or fidelity-inconsistent. Multilevel regression models were used to estimate the relationship between adaptations and implementation and participant outcomes. Results: Of 37 sites interviewed, 28 sites (76%) made adaptations to InSHAPE ( M = 2.1, SD = 1.3). Sixteen sites (43%) made fidelity-consistent adaptations, while 22 (60%) made fidelity-inconsistent adaptations. The number of fidelity-inconsistent adaptations was negatively associated with InSHAPE fidelity scores (β = −4.29; p < .05). A greater number of adaptations were associated with significantly higher odds of participant-level cardiovascular risk reduction (odds ratio [ OR] = 1.40; confidence interval [CI] = [1.08, 1.80]; p < .05). With respect to the type of adaptation, we found a significant positive association between the number of fidelity-inconsistent adaptations and cardiovascular risk reduction ( OR = 1.59; CI = [1.01, 2.51]; p < .05). This was largely explained by the fidelity-inconsistent adaptation of holding exercise sessions at the mental health agency versus a fitness facility in the community (a core form of InSHAPE) ( OR = 2.52; 95% CI = [1.11, 5.70]; p < .05). Conclusions: This research suggests that adaptations to an evidence-based lifestyle program were common during implementation in real-world mental health practice settings even when fidelity was monitored and reinforced through implementation interventions. Results suggest that adaptations, including those that are fidelity-inconsistent, can be positively associated with improved participant outcomes when they provide a potential practical advantage while maintaining the core function of the intervention. Plain language abstract: Treatments that have been proven to work in research studies are not always one-size-fits-all. In real-world clinical settings where people receive mental health care, sometimes there are good reasons to change certain things about a treatment. For example, a particular treatment might not fit well in a specific clinic or cultural context, or it might not meet the needs of specific patient groups. We studied adaptations to an evidence-based practice (InSHAPE) targeting obesity in persons with serious mental illness made by teams implementing the program in routine mental health care settings. We learned that adaptations to InSHAPE were common, and that an adaptation that model experts initially viewed as inconsistent with fidelity to the model turned out to have a positive impact on participant health outcomes. The results of this study may encourage researchers and model experts to work collaboratively with mental health agencies and clinicians implementing evidence-based practices to consider allowing for and guiding adaptations that provide a potential practical advantage while maintaining the core purpose of the intervention.


Author(s):  
Emma M Parrish ◽  
Snigdha Kamarsu ◽  
Philip D Harvey ◽  
Amy Pinkham ◽  
Colin A Depp ◽  
...  

Abstract Smartphone-based ecological mobile cognitive tests (EMCTs) can measure cognitive abilities in the real world, complementing traditional neuropsychological assessments. We evaluated the validity of an EMCT of recognition memory designed for use with people with serious mental illness, as well as relevant contextual influences on performance. Participants with schizophrenia (SZ), schizoaffective disorder, and bipolar disorder (BD) completed in-lab assessments of memory (Hopkins Verbal Learning Test, HVLT), other cognitive abilities, functional capacity, and symptoms, followed by 30 days of EMCTs during which they completed our Mobile Variable Difficulty List Memory Test (VLMT) once every other day (3 trials per session). List length on the VLMT altered between 6, 12, and 18 items. On average, participants completed 75.3% of EMCTs. Overall performance on VLMT 12 and 18 items was positively correlated with HVLT (ρ = 0.52, P &lt; .001). People with BD performed better on the VLMT than people with SZ. Intraindividual variability on the VLMT was more specifically associated with HVLT than nonmemory tests and not associated with symptoms. Performance during experienced distraction, low effort, and out of the home location was reduced yet still correlated with the in-lab HVLT. The VLMT converged with in-lab memory assessment, demonstrating variability within person and by different contexts. Ambulatory cognitive testing on participants’ personal mobile devices offers more a cost-effective and “ecologically valid” measurement of real-world cognitive performance.


2018 ◽  
Vol Volume 10 ◽  
pp. 573-585 ◽  
Author(s):  
Jason Shafrin ◽  
Katalin Bognar ◽  
Katie Everson ◽  
Michelle Brauer ◽  
Darius N Lakdawalla ◽  
...  

2009 ◽  
Vol 32 (3) ◽  
pp. 299-308 ◽  
Author(s):  
Allen E. Thornton ◽  
Hayley Kristinsson ◽  
Vanessa G. DeFreitas ◽  
Wendy Loken Thornton

F1000Research ◽  
2018 ◽  
Vol 7 ◽  
pp. 210 ◽  
Author(s):  
Richard Jackson ◽  
Rashmi Patel ◽  
Sumithra Velupillai ◽  
George Gkotsis ◽  
David Hoyle ◽  
...  

Background: Deep Phenotyping is the precise and comprehensive analysis of phenotypic features in which the individual components of the phenotype are observed and described. In UK mental health clinical practice, most clinically relevant information is recorded as free text in the Electronic Health Record, and offers a granularity of information beyond what is expressed in most medical knowledge bases. The SNOMED CT nomenclature potentially offers the means to model such information at scale, yet given a sufficiently large body of clinical text collected over many years, it is difficult to identify the language that clinicians favour to express concepts. Methods: By utilising a large corpus of healthcare data, we sought to make use of semantic modelling and clustering techniques to represent the relationship between the clinical vocabulary of internationally recognised SMI symptoms and the preferred language used by clinicians within a care setting. We explore how such models can be used for discovering novel vocabulary relevant to the task of phenotyping Serious Mental Illness (SMI) with only a small amount of prior knowledge.  Results: 20 403 terms were derived and curated via a two stage methodology. The list was reduced to 557 putative concepts based on eliminating redundant information content. These were then organised into 9 distinct categories pertaining to different aspects of psychiatric assessment. 235 concepts were found to be expressions of putative clinical significance. Of these, 53 were identified having novel synonymy with existing SNOMED CT concepts. 106 had no mapping to SNOMED CT. Conclusions: We demonstrate a scalable approach to discovering new concepts of SMI symptomatology based on real-world clinical observation. Such approaches may offer the opportunity to consider broader manifestations of SMI symptomatology than is typically assessed via current diagnostic frameworks, and create the potential for enhancing nomenclatures such as SNOMED CT based on real-world expressions.


2020 ◽  
Author(s):  
Kristina Schnitzer ◽  
Corrine Cather ◽  
Vanya Zvonar ◽  
Alyson Dechert ◽  
Rachel Plummer ◽  
...  

BACKGROUND In a prior study, participation in a 16-week, reverse integrated care, group behavioral and educational intervention for individuals with diabetes and serious mental illness was associated with improved glycemic control (HbA1C) and body mass index (BMI). In order to inform future implementation efforts, more information about the effective components of the intervention is needed. OBJECTIVE The goal of this study was to identify aspects of the intervention participants reported were helpful and to evaluate predictors of outcome. METHODS This study involved qualitative evaluation and post-hoc quantitative analysis of a prior intervention. Qualitative data were collected using semi-structured interviews with 24 of 35 individuals (69%) who attended one or more group sessions and 9 of 26 individuals (35%) who consented but attended no sessions. Quantitative mixed effects modeling was performed to test whether improved diabetes knowledge, diet and exercise, or higher group attendance predicted improved HbA1C and BMI. These interview and modeling outcomes were combined using a mixed methods case study framework and integrated thematically. RESULTS In qualitative interviews, participants identified application of health-related knowledge gained to real world situations, accountability for goals, positive reinforcement and group support, and increased confidence to prioritize health goals as factors contributing to success of the behavioral intervention. Improved diabetes knowledge was associated with reduced BMI (=-1.27, SD=0.40, P=0.003). No quantitative variables examined were significantly associated with improved HbA1C. CONCLUSIONS In this mixed methods analysis of predictors of success in a behavioral diabetes management program, group participants highlighted the value of positive reinforcement and group support, accountability for goals set, and real-world application of health-related knowledge gained. Improved diabetes knowledge was associated with weight loss. CLINICALTRIAL


F1000Research ◽  
2018 ◽  
Vol 7 ◽  
pp. 210 ◽  
Author(s):  
Richard Jackson ◽  
Rashmi Patel ◽  
Sumithra Velupillai ◽  
George Gkotsis ◽  
David Hoyle ◽  
...  

Background: Deep Phenotyping is the precise and comprehensive analysis of phenotypic features, where the individual components of the phenotype are observed and described. In UK mental health clinical practice, most clinically relevant information is recorded as free text in the Electronic Health Record, and offers a granularity of information beyond that expressed in most medical knowledge bases. The SNOMED CT nomenclature potentially offers the means to model such information at scale, yet given a sufficiently large body of clinical text collected over many years, it’s difficult to identify the language that clinicians favour to express concepts. Methods: Vector space models of language seek to represent the relationship between words in a corpus in terms of cosine distance between a series of vectors. When utilising a large corpus of healthcare data and combined with appropriate clustering techniques and manual curation, we explore how such models can be used for discovering vocabulary relevant to the task of phenotyping Serious Mental Illness (SMI) with only a small amount of prior knowledge. Results: 20 403 n-grams were derived and curated via a two stage methodology. The list was reduced to 557 putative concepts based on eliminating redundant information content. These were then organised into 9 distinct categories pertaining to different aspects of psychiatric assessment. 235 (42%) concepts were found to be depictions of putative clinical significance. Of these, 53 (10%) were identified having novel synonymy with existing SNOMED CT concepts. 106 (19%) had no mapping to SNOMED CT. Conclusions: We demonstrate a scalable approach to discovering new depictions of SMI symptomatology based on real world clinical observation. Such approaches may offer the opportunity to consider broader manifestations of SMI symptomatology than is typically assessed via current diagnostic frameworks, and create the potential for enhancing nomenclatures such as SNOMED CT based on real world depictions.


Sign in / Sign up

Export Citation Format

Share Document