scholarly journals Development of the Huddle Observation Tool for structured case management discussions to improve situation awareness on inpatient clinical wards

2017 ◽  
Vol 27 (5) ◽  
pp. 365-372 ◽  
Author(s):  
Julian Edbrooke-Childs ◽  
Jacqueline Hayes ◽  
Evelyn Sharples ◽  
Dawid Gondek ◽  
Emily Stapley ◽  
...  
Keyword(s):  
Case Management ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Situation Awareness ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Weighted Kappa ◽  
Observational Assessment ◽  
Rater Reliability ◽  
Team Processes

Background‘Situation Awareness For Everyone’ (SAFE) was a 3-year project which aimed to improve situation awareness in clinical teams in order to detect potential deterioration and other potential risks to children on hospital wards. The key intervention was the ‘huddle’, a structured case management discussion which is central to facilitating situation awareness. This study aimed to develop an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle.MethodsA cross-sectional observational design was used to psychometrically develop the ‘Huddle Observation Tool’ (HOT) over three phases using standardised psychometric methodology. Huddles were observed across four NHS paediatric wards participating in SAFE by five researchers; two wards within specialist children hospitals and two within district general hospitals, with location, number of beds and length of stay considered to make the sample as heterogeneous as possible. Inter-rater reliability was calculated using the weighted kappa and intraclass correlation coefficient.ResultsInter-rater reliability was acceptable for the collaborative culture (weighted kappa=0.32, 95% CI 0.17 to 0.42), environment items (weighted kappa=0.78, 95% CI 0.52 to 1) and total score (intraclass correlation coefficient=0.87, 95% CI 0.68 to 0.95). It was lower for the structure and risk management items, suggesting that these were more variable in how observers rated them. However, agreement on the global score for huddles was acceptable.ConclusionWe developed an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle. Future research should examine whether observational evaluations of huddles are associated with other indicators of safety on clinical wards (eg, safety climate and incidents of patient harm), and whether scores on the HOT are associated with improved situation awareness and reductions in deterioration and adverse events in clinical settings, such as inpatient wards.

scholarly journals Intra-rater reliability of transversus abdominis measurement by a novice examiner: Comparison of “freehand” to “probe force device” method of real-time ultrasound imaging

Ultrasound ◽  
2019 ◽  
Vol 27 (3) ◽  
pp. 156-166 ◽  
Author(s):  
Vanessa L Kennedy ◽  
Carol A Flavell ◽  
Kenji Doma
Keyword(s):  
Measurement Error ◽  
Real Time ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Coefficient Of Variation ◽  
Repeated Measures ◽  
Intraclass Correlation ◽  
Transversus Abdominis ◽  
Rater Reliability ◽  
Transverse Abdominis

A “free hand” real-time-ultrasound method is commonly applied to measure transversus abdominis. Potentially, this increases transversus abdominis measurement error due to uncontrolled variability in probe to skin force, inclination, and roll, particularly for novice examiners. This single-group repeated-measures reliability study compared the intra-rater reliability of transversus abdominis thickness and activation measurement by a novice examiner between free hand and a standardized probe force device method. The examiner captured ultrasound videos of transversus abdominis in a single session in healthy participants ( n =  33). Free hand ultrasound featured uncontrolled probe force, inclination, and roll, while probe force device method ultrasound standardized these parameters. Images of transversus abdominis at rest and contracted were measured and transversus abdominis activation calculated. Intraclass correlation coefficient, coefficient of variation, standard error of measurement, and worthwhile differences were calculated. The probe force device method resulted in greater reliability (intraclass correlation coefficient = 0.75–0.96) and lower measurement error (coefficient of variation = 8.89–28.7%) compared to free hand (intraclass correlation coefficient = 0.63–0.93; coefficient of variation = 6.52–29.4%). Reliability was good for all measurements except free hand TrA-C, which was moderate. TrA-C had the lowest reliability, followed by contracted thickness of the transverse abdominis, with resting thickness of the transverse abdominis being highest. Worthwhile differences were lower using a probe force device method versus free hand for resting thickness of the transverse abdominis and contracted thickness of the transverse abdominis and similar for TrA-C. Standardization using probe force device method ultrasound to measure transversus abdominis improved intra-rater reliability in a novice examiner. Use of a probe force device method is recommended to improve reliability through reduced sources of measurement error. Probe force device method intra- and inter-rater reliability in examiners of varying experience, in clinical populations, and to visualize other structures merits exploration.

Learning how to Differ: Agreement and Reliability Statistics in Psychiatry

1995 ◽  
Vol 40 (2) ◽  
pp. 60-66 ◽  
Author(s):  
L. Streiner David
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Weighted Kappa ◽  
The Other ◽  
Cohen’S Kappa ◽  
Cohen's Kappa ◽  
The Subject

Whenever two or more raters evaluate a patient or student, it may be necessary to determine the degree to which they assign the same label or rating to the subject. The major problem in deciding which statistic to use is the plethora of different techniques which are available. This paper reviews some of the more commonly used techniques, such as Raw Agreement, Cohen's kappa and weighted kappa, and shows that, in most circumstances, they can all be replaced by the intraclass correlation coefficient (ICC). This paper also shows how the ICC can be used in situations where the other statistics cannot be used and how to select the best subset of raters.

scholarly journals Validity and reliability of smartphone-based application for chronic ankle instability

2021 ◽  
Vol 28 (9) ◽  
pp. 1-10
Author(s):  
Taelim Yoon ◽  
Jihyun Lee
Keyword(s):  
Medical Device ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Ankle Instability ◽  
Intraclass Correlation ◽  
Validity And Reliability ◽  
Rater Reliability ◽  
Eyes Closed ◽  
Eyes Open ◽  
Cumberland Ankle Instability Tool

Background/aims Ankle instability is one of the most common injuries that can occur during everyday life, sports and exercise. Recently, smartphone accelerometers have been used to measure single leg balance associated with ankle instability, because they are easy to use, inexpensive and can be used in small spaces. Thus, the purpose of this study was to introduce and investigate the intra- and inter-rater reliability of the smartphone accelerometer when assessing ankle instability. Methods A total of 26 individuals who had ankle instability were recruited. The single leg stance balance was measured using a smartphone accelerometer (Accelerometer application) and a force platform (I-Balance) for 5 seconds with their eyes open or their eyes closed. Results In the eyes open position, intra-rater reliability of the smartphone accelerometer was excellent for both raters (intraclass correlation coefficient: 0.87–0.90); and the inter-rater reliability was moderate (intraclass correlation coefficient: 0.71). In the eyes closed position, the intra-rater reliability of the smartphone accelerometer was excellent for both raters (intraclass correlation coefficient: 0.90–0.93); the inter-rater reliability was good (intraclass correlation coefficient: 0.82). Additionally, there were fair positive correlations between the smartphone accelerometer and the Cumberland Ankle Instability Tool, and between the smartphone accelerometer and I-Balance (r=0.33, 0.30 respectively). Conclusions The present study demonstrated excellent intra-rater reliabilities of two raters and moderate to good inter-rater reliabilities. The smartphone accelerometer offers several important advantages as a potential portable medical device to assess ankle instability accurately. Although there was a positive correlation, the relationships between the smartphone accelerometer and Cumberland Ankle Instability Tool and that between the smartphone accelerometer and I-Balance were fair. Future studies should investigate the validity of the smartphone accelerometer as a portable medical device for determining ankle instability.

scholarly journals Validity and Reliability of the Persian Version of Language Screening Test (LAST) for Patients in the Acute Phase of Stroke

2020 ◽  
Vol 3 (1) ◽  
pp. 91-100
Author(s):  
Seyyede Zohreh Mousavi ◽  
◽  
Reyhaneh Jafari ◽  
Saman Maroufizadeh3 ◽  
Mohammad Moez Shahramnia ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Acute Phase ◽  
Screening Test ◽  
Intraclass Correlation ◽  
Chronic Phase ◽  
Weighted Kappa ◽  
Screening Tests ◽  
Rater Reliability ◽  
Persian Version ◽  
Language Screening

Background & Objectives: Aphasia is one of the most common consequences of a stroke; thus, screening tests for early diagnosis of the problem are necessary when dealing with aphasia patients. One of these screening tests is the Language Screening Test (LAST). The purpose of this study was to translate, validate, and utilize this test in the Persian language for patients after stroke. Methods: The original version of LAST was translated into Persian, and then administrated on 100 patients in the acute phase by two examiners at the patient’s bedside in order to check the inter-rater reliability. To assess the agreement between the two forms (a and b) of the LAST, Concordance Correlation Coefficient (CCC), weighted Kappa, and Intraclass Correlation Coefficient (ICC) were used. Also, the Persian version of LAST and the Western Aphasia Battery (WAB) were performed at the chronic phase with two independent examiners with blind scoring. Results: Inter-rater reliability between Rater 1 and Rater 2 on LAST-a and LAST-b score were very good for both phases. The CCC for LAST-a and LAST-b, respectively, were 0.874 and 0.865 for the acute phase and 0.923 and 0.927 for the chronic phase. The weighted Kappa for LAST-a and LAST-b, respectively, were 0.750 and 0.740 for the acute phase, and 0.822 and 0.846 for the chronic phase. Conclusion: The obtained results showed that LAST is a very simple, fast, and valid test and can be used as a reliable tool in stroke patients. Lack of cultural and language dependency are the advantages of using this test.

scholarly journals The Inventory of Physical Activity Barriers for Adults 50 Years and Older: Refinement and Validation

2021 ◽  
Vol 5 (Supplement_1) ◽  
pp. 789-789
Author(s):  
Mariana Wingood ◽  
Salene Jones ◽  
Nancy Gell ◽  
Denise Peters ◽  
Jennifer Brach
Keyword(s):  
Physical Activity ◽  
Factor Analysis ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Community Dwelling ◽  
Cross Sectional ◽  
Retest Reliability ◽  
Test Retest Reliability

Abstract Addressing physical activity (PA) barriers is an essential component of increasing PA among the 56-73% of community-dwelling adults 50 years and older who are not performing the recommended 150 minutes of moderate-to-vigorous PA. As there is no feasible, multi-factorial tool to assess PA barriers among this population, we developed and validated a PA barrier assessment tool called the Inventory of Physical Activity Barriers (IPAB). We collected cross-sectional data on 503 adults (mean age 70.1), with 79 participants completing the scale twice for test-retest reliability and 64 completing a cross-over design examining the ability to use two administration formats interchangeably. Our analyses consisted of exploratory and confirmatory factor analysis, Cronbach alpha, intraclass correlation coefficient, Bland-Altman Plot, and t-tests. Using factor analysis, we identified and confirmed an eight-factor solution consisting of 27 items. The 27-item IPAB is internally consistent (alpha= 0.91), has a high test-retest reliability (intraclass correlation coefficient=0.99), and can differentiate between individuals who meet the recommended levels of PA and those who do not (p < 0.001). The IPAB scores ranged between 1.00-3.11 for the paper format (mean=1.78) and 1.07-3.48 for the electronic format (mean=1.78), with no statistical difference between the paper and electronic administration formats (p=0.94), resulting in the conclusion that the two administration formats can be used interchangeably. Participant feedback illustrates that the IPAB is easy to use, has clear instruction, and is an appropriate length. The newly validated IPAB scale can be used to develop individualized PA interventions that address PA barriers among patients 50 years and older.

User testing of the psychometric properties of pictorial-based disability assessment Longshi Scale by healthcare professionals and non-professionals: a Chinese study in Shenzhen

2019 ◽  
Vol 33 (9) ◽  
pp. 1479-1491 ◽  
Author(s):  
Yulong Wang ◽  
Shanshan Guo ◽  
Jiejiao Zheng ◽  
Qing Mei Wang ◽  
Yuling Zhang ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Barthel Index ◽  
Public Hospital ◽  
Healthcare Professionals ◽  
Intraclass Correlation ◽  
Random Effect ◽  
Second Phase ◽  
Spearman Correlation ◽  
Rater Reliability

Objective:The aim of this study was to validate a novel pictorial-based Longshi Scale for evaluating a patient’s disability by healthcare professionals and non-professionals.Design:Prospective study.Setting:Rehabilitation departments from a grade A, class 3 public hospital, a grade B, class 2 public hospital, and a private hospital and seven community rehabilitation centers.Subjects:A total of 618 patients and 251 patients with functional disabilities were recruited in a two-phase study, respectively.Main measures:Outcome measure: pictorial scale of activities of daily living (ADLs, Longshi Scale). Reference measure: Barthel Index. The Spearman correlation coefficient was used to analyze the validity of Longshi Scale against Barthel Index.Results:In phase 1 study, from March 2016 to August 2016, the results demonstrated that the Longshi Scale was both reliable and valid (intraclass correlation coefficient based on two-way random effect (ICC2,1) = 0.877–0.974 for intra-rater reliability; ICC2,1= 0.928–0.979; κ = 0.679–1.000 for inter-rater reliability; intraclass correlation coefficient based on one-way random effect (ICC1,1) = 0.921–0.984 for test–retest reliability and Spearman correlation coefficient = 0.836–0.899). In the second phase, in March 2018, results further demonstrated that the Longshi Scale had good inter-rater and intra-rater reliability among healthcare professionals and non-professionals including therapists, interns, and personal care aids (ICC1,1= 0.822–0.882 on Day 1; ICC1,1= 0.842–0.899 on Day 7 for inter-rater reliability). In addition, the Longshi Scale decreased assessment time significantly, compared with the Barthel Index assessment ( P < 0.01).Conclusion:The Longshi Scale could potentially provide an efficient way for healthcare professionals and non-professionals who may have minimal training to assess the ADLs of functionally disabled patients.

scholarly journals Development and reliability of the Korean version of the Feeding Abilities Assessment

2019 ◽  
Vol 32 (1) ◽  
pp. 69-74
Author(s):  
Seul Gi Koo ◽  
Hae Yean Park ◽  
Jongbae Kim ◽  
Areum Han
Keyword(s):  
Correlation Coefficient ◽  
Internal Consistency ◽  
Content Validity ◽  
Assessment Tool ◽  
High Reliability ◽  
Intraclass Correlation ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Korean Version ◽  
Test Retest Reliability

Objective The purpose of this study is to introduce a standardised assessment tool by verifying the reliability of the translated Korean version of the Feeding Abilities Assessment (K-FAA), which was developed to suit Korean culture. Methods The research subjects were 65 patients with dementia living in nursing homes. The K-FAA was completed by verifying the suitability of translation and reverse translation. The validity of the K-FAA was established through content validity, while its reliability was analysed based on internal consistency reliability for the items, test–retest reliability and inter-rater reliability. Results The content validity index determined, based on the assessment of professors, occupational therapists, and nurses, was more than .70. Cronbach’s α was more than .929, showing good internal consistency. A test–retest reliability of .884 was derived using Pearson’s correlation coefficient (p < .01), and an inter-rater reliability of .800 was derived using the kappa coefficients; intraclass correlation coefficient was .897, which also indicated good reliability. Conclusion The K-FAA was modified to fit the Korean domestic situation, and this assessment had high reliability. Therefore, K-FAA can evaluate the feeding ability of patients with dementia. Future studies should focus on providing evidence-based data to maintain or supplement the feeding ability of patients with dementia in Korea.

Development and Validation of a Questionnaire to Measure Attitude change in Health Professionals after Completion of an Aboriginal Health and Cultural Safety Training Programme

2017 ◽  
Vol 48 (01) ◽  
pp. 24-38 ◽  
Author(s):  
Courtney Ryder ◽  
Tamara Mackean ◽  
Shahid Ullah ◽  
Heather Burton ◽  
Heather Halls ◽  
...  
Keyword(s):  
Health Professional ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Attitude Change ◽  
Focal Point ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Weighted Kappa ◽  
Aboriginal Health ◽  
Cultural Safety

Socially accountable health curricula, designed to decrease Aboriginal health inequities through the transformation of health professional students into culturally safe practitioners, has become a focal point for health professional programmes. Despite this inclusion in health curricula there remains the question of how to best assess students in this area in relation to the concept, of cultural safety and transformative unlearning, to facilitate attitudinal change. To address this question, this study developed a research questionnaire to measure thematic areas of transformative unlearning, cultural safety and critical thinking in Aboriginal Health for application on undergraduate and postgraduate students and faculty staff. The Likert-scale questionnaire was developed and validated through face and content validity. Test–retest methodology was utilised to determine stability and reliability of the questionnaire with 40 participants. The extent of agreement and reliability were determined through weighted kappa and intraclass correlation coefficient. Exploratory factor analysis was calculated to determine construct validity for questionnaire items. For the overall population subset the tool met good standards of reliability and validity, with 11 of the 15 items reaching moderate agreement (κ &gt; 0.6) and an intraclass correlation coefficient of 0.72, suggesting substantial agreement. Cronbach's alpha was calculated above 0.7 for the thematic areas. The majority of items provided high factor loadings, low loading items will be reviewed to strengthen the tool, where validations of the revised tool with a larger cohort will allow future use to compare and determine effective teaching methodologies in Aboriginal health and cultural safety curricula.

scholarly journals Intra- and inter-rater reliability in a comparative study of cross-sectional and spiral computed tomography pelvimetry methods

2019 ◽  
Vol 8 (6) ◽  
pp. 205846011985518 ◽  
Author(s):  
Erika Phexell ◽  
Anna Åkesson ◽  
Marcus Söderberg ◽  
Anetta Bolejko
Keyword(s):  
Computed Tomography ◽  
Radiation Dose ◽  
Confidence Interval ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Measurement Reliability ◽  
Cross Sectional ◽  
Rater Reliability ◽  
Short Spiral

Background Different low-dose computed tomography (CT) pelvimetry methods can be used to evaluate the size of birth canal before delivery. CT pelvimetry might generate an acceptable low fetal radiation dose but its measurement accuracy is unknown. Purpose To investigate intra- and inter-rater measurement reliability of cross-sectional and two spiral CT pelvimetry methods: standard spiral and short spiral. Material and Methods Ten individuals (age ≥60 years, body mass index ≥30 kg/m2) having a CT scan of the abdomen also had CT pelvimetry scans. Three radiologists made independent measurements of each pelvimetry method on two occasions and also in consensus for a reference pelvimetry computed from the standard-dose CT scan of the abdomen. Inter- and intra-rater reliability was analyzed by intraclass correlation coefficient. Results Measurements in the short spiral pelvimetry demonstrated excellent intra- and inter-rater reliability, intraclass correlation coefficient ≥0.93, and good to excellent 95% confidence interval 0.87–0.99. Corresponding results of the standard spiral and cross-sectional pelvimetry showed good to excellent intraclass correlation coefficient ≥0.85 and ≥0.76, and 95% confidence interval was least good and moderate 0.73–0.98 and 0.59–0.97, respectively. Intraclass correlation coefficient between reference pelvimetry and other CT methods showed analogous results. Conclusion The short spiral pelvimetry demonstrated high and best reliability in comparison to other methods. Standard spiral method showed also good measurement reliability but the short spiral pelvimetry generates lower fetal radiation dose. This method might be suitable for measurements at narrow pelvis. Patient acceptance and attitude to CT pelvimetry should be investigated.

Sign in / Sign up

Export Citation Format

Share Document