User testing of the psychometric properties of pictorial-based disability assessment Longshi Scale by healthcare professionals and non-professionals: a Chinese study in Shenzhen

2019 ◽  
Vol 33 (9) ◽  
pp. 1479-1491 ◽  
Author(s):  
Yulong Wang ◽  
Shanshan Guo ◽  
Jiejiao Zheng ◽  
Qing Mei Wang ◽  
Yuling Zhang ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Barthel Index ◽  
Public Hospital ◽  
Healthcare Professionals ◽  
Intraclass Correlation ◽  
Random Effect ◽  
Second Phase ◽  
Spearman Correlation ◽  
Rater Reliability

Objective:The aim of this study was to validate a novel pictorial-based Longshi Scale for evaluating a patient’s disability by healthcare professionals and non-professionals.Design:Prospective study.Setting:Rehabilitation departments from a grade A, class 3 public hospital, a grade B, class 2 public hospital, and a private hospital and seven community rehabilitation centers.Subjects:A total of 618 patients and 251 patients with functional disabilities were recruited in a two-phase study, respectively.Main measures:Outcome measure: pictorial scale of activities of daily living (ADLs, Longshi Scale). Reference measure: Barthel Index. The Spearman correlation coefficient was used to analyze the validity of Longshi Scale against Barthel Index.Results:In phase 1 study, from March 2016 to August 2016, the results demonstrated that the Longshi Scale was both reliable and valid (intraclass correlation coefficient based on two-way random effect (ICC2,1) = 0.877–0.974 for intra-rater reliability; ICC2,1= 0.928–0.979; κ = 0.679–1.000 for inter-rater reliability; intraclass correlation coefficient based on one-way random effect (ICC1,1) = 0.921–0.984 for test–retest reliability and Spearman correlation coefficient = 0.836–0.899). In the second phase, in March 2018, results further demonstrated that the Longshi Scale had good inter-rater and intra-rater reliability among healthcare professionals and non-professionals including therapists, interns, and personal care aids (ICC1,1= 0.822–0.882 on Day 1; ICC1,1= 0.842–0.899 on Day 7 for inter-rater reliability). In addition, the Longshi Scale decreased assessment time significantly, compared with the Barthel Index assessment ( P < 0.01).Conclusion:The Longshi Scale could potentially provide an efficient way for healthcare professionals and non-professionals who may have minimal training to assess the ADLs of functionally disabled patients.

scholarly journals High inter-rater reliability of Japanese bedriddenness ranks and cognitive function scores: a hospital-based prospective observational study

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Masaki Tago ◽  
Naoko E. Katsuki ◽  
Shizuka Yaita ◽  
Eiji Nakatani ◽  
Shun Yamashita ◽  
...  
Keyword(s):  
Cognitive Function ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Barthel Index ◽  
Prospective Observational Study ◽  
Intraclass Correlation ◽  
Kappa Coefficient ◽  
Concordance Rate ◽  
Rater Reliability ◽  
Katz Index

Abstract Background The statistical validities of the official Japanese classifications of activities of daily living (ADLs), including bedriddenness ranks (BR) and cognitive function scores (CFS), have yet to be assessed. To this aim, we evaluated the ability of BR and CFS to assess ADLs using inter-rater reliability and criterion-related validity. Methods New inpatients aged ≥75 years were enrolled in this hospital-based prospective observational study. BR and CFS were assessed once by an attending nurse, and then by a social worker/medical clerk. We evaluated inter-rater reliability between different professions by calculating the concordance rate, kappa coefficient, Cronbach’s α, and intraclass correlation coefficient. We also estimated the relationship of the Barthel Index and Katz Index with the BR and CFS using Spearman’s correlation coefficients. Results For the 271 patients enrolled, BR at the first assessment revealed 66 normal, 10 of J1, 15 of J2, 18 of A1, 31 of A2, 37 of B1, 35 of B2, 22 of C1, and 32 of C2. The concordance rate between the two BR assessments was 68.6%, with a kappa coefficient of 0.61, Cronbach’s α of 0.91, and an intraclass correlation coefficient of 0.83, thus showing good inter-rater reliability. BR was negatively correlated with the Barthel Index (r = − 0.848, p < 0.001) and Katz Index (r = − 0.820, p < 0.001), showing justifiable criterion-related validity. Meanwhile, CFS at the first assessment revealed 92 normal, 47 of 1, 19 of 2a, 30 of 2b, 60 of 3a, 8 of 3b, 8 of 4, and 0 of M. The concordance rate between the two CFS assessments was 70.1%, with a kappa coefficient of 0.62, Cronbach’s α of 0.87, and an intraclass correlation coefficient 0.78, thus also showing good inter-rater reliability. CFS was negatively correlated with the Barthel Index (r = − 0.667, p < 0.001) and Katz Index (r = − 0.661, p < 0.001), showing justifiable criterion-related validity. Conclusions BR and CFS could be reliable and easy-to-use grading scales of ADLs in acute clinical practice or large-scale screening, with high inter-rater reliabilities among different professions and significant correlations with well-established, though complicated to use, instruments to assess ADLs. Trial registration UMIN000041051 (2020/7/10).

scholarly journals Equation Córdoba: A Simplified Method for Estimation of Body Fat (ECORE-BF)

2019 ◽  
Vol 16 (22) ◽  
pp. 4529 ◽  
Author(s):  
Molina-Luque ◽  
Romero-Saldaña ◽  
Álvarez-Fernández ◽  
Bennasar-Veny ◽  
Álvarez-López ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Body Fat ◽  
Intraclass Correlation ◽  
The Body ◽  
Second Phase ◽  
Fat Percentage ◽  
Daily Work ◽  
Two Phases ◽  
New Equation

Background: Many methods for measuring body fat have been developed, but applications in clinical settings are limited. For this reason, researchers have tried to identify different formulas for its estimation but most of are hard to incorporate into daily work due to the variability in population and difficulty of use. The aim of this study was to develop and validate a new equation for the simplified estimation of body fat using the Clínica Universidad de Navarra – Body Adiposity Estimator (CUN-BAE) as a reference. Methods: This research was conducted in two phases. In the first, the new body fat estimation equation was developed. The developed equation was validated in the second phase. Pearson’s linear correlation, raw and adjusted linear regressions, the intraclass correlation coefficient, and Bland–Altman graphs were used. Results: The variables that best adjusted the body fat percentage were age, sex, and the Napierian logarithm of Body Mass Index (LnBMI), forming the Equation Córdoba for Estimation of Body Fat (ECORE-BF) model. In its validation, the model presented correlation values of 0.994, an intraclass correlation coefficient of 0.960, with the Bland–Altman graph indicating means differences of 1.82 with respect to the estimation with the CUN-BAE. Nevertheless, although the aim was to simplify the CUN-BAE, the main limitation of this study is that a gold standard, such as air displacement plethysmography (ADP) or dual-energy X-ray absorptiometry (DXA), was not used. Conclusions: The proposed equation (ECORE-BF) simplified the CUN-BAE and provided a precise method, respecting the principle of parsimony, for the calculation of body fat.

scholarly journals Intra-rater reliability of transversus abdominis measurement by a novice examiner: Comparison of “freehand” to “probe force device” method of real-time ultrasound imaging

Ultrasound ◽  
2019 ◽  
Vol 27 (3) ◽  
pp. 156-166 ◽  
Author(s):  
Vanessa L Kennedy ◽  
Carol A Flavell ◽  
Kenji Doma
Keyword(s):  
Measurement Error ◽  
Real Time ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Coefficient Of Variation ◽  
Repeated Measures ◽  
Intraclass Correlation ◽  
Transversus Abdominis ◽  
Rater Reliability ◽  
Transverse Abdominis

A “free hand” real-time-ultrasound method is commonly applied to measure transversus abdominis. Potentially, this increases transversus abdominis measurement error due to uncontrolled variability in probe to skin force, inclination, and roll, particularly for novice examiners. This single-group repeated-measures reliability study compared the intra-rater reliability of transversus abdominis thickness and activation measurement by a novice examiner between free hand and a standardized probe force device method. The examiner captured ultrasound videos of transversus abdominis in a single session in healthy participants ( n =  33). Free hand ultrasound featured uncontrolled probe force, inclination, and roll, while probe force device method ultrasound standardized these parameters. Images of transversus abdominis at rest and contracted were measured and transversus abdominis activation calculated. Intraclass correlation coefficient, coefficient of variation, standard error of measurement, and worthwhile differences were calculated. The probe force device method resulted in greater reliability (intraclass correlation coefficient = 0.75–0.96) and lower measurement error (coefficient of variation = 8.89–28.7%) compared to free hand (intraclass correlation coefficient = 0.63–0.93; coefficient of variation = 6.52–29.4%). Reliability was good for all measurements except free hand TrA-C, which was moderate. TrA-C had the lowest reliability, followed by contracted thickness of the transverse abdominis, with resting thickness of the transverse abdominis being highest. Worthwhile differences were lower using a probe force device method versus free hand for resting thickness of the transverse abdominis and contracted thickness of the transverse abdominis and similar for TrA-C. Standardization using probe force device method ultrasound to measure transversus abdominis improved intra-rater reliability in a novice examiner. Use of a probe force device method is recommended to improve reliability through reduced sources of measurement error. Probe force device method intra- and inter-rater reliability in examiners of varying experience, in clinical populations, and to visualize other structures merits exploration.

scholarly journals Validity and reliability of smartphone-based application for chronic ankle instability

2021 ◽  
Vol 28 (9) ◽  
pp. 1-10
Author(s):  
Taelim Yoon ◽  
Jihyun Lee
Keyword(s):  
Medical Device ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Ankle Instability ◽  
Intraclass Correlation ◽  
Validity And Reliability ◽  
Rater Reliability ◽  
Eyes Closed ◽  
Eyes Open ◽  
Cumberland Ankle Instability Tool

Background/aims Ankle instability is one of the most common injuries that can occur during everyday life, sports and exercise. Recently, smartphone accelerometers have been used to measure single leg balance associated with ankle instability, because they are easy to use, inexpensive and can be used in small spaces. Thus, the purpose of this study was to introduce and investigate the intra- and inter-rater reliability of the smartphone accelerometer when assessing ankle instability. Methods A total of 26 individuals who had ankle instability were recruited. The single leg stance balance was measured using a smartphone accelerometer (Accelerometer application) and a force platform (I-Balance) for 5 seconds with their eyes open or their eyes closed. Results In the eyes open position, intra-rater reliability of the smartphone accelerometer was excellent for both raters (intraclass correlation coefficient: 0.87–0.90); and the inter-rater reliability was moderate (intraclass correlation coefficient: 0.71). In the eyes closed position, the intra-rater reliability of the smartphone accelerometer was excellent for both raters (intraclass correlation coefficient: 0.90–0.93); the inter-rater reliability was good (intraclass correlation coefficient: 0.82). Additionally, there were fair positive correlations between the smartphone accelerometer and the Cumberland Ankle Instability Tool, and between the smartphone accelerometer and I-Balance (r=0.33, 0.30 respectively). Conclusions The present study demonstrated excellent intra-rater reliabilities of two raters and moderate to good inter-rater reliabilities. The smartphone accelerometer offers several important advantages as a potential portable medical device to assess ankle instability accurately. Although there was a positive correlation, the relationships between the smartphone accelerometer and Cumberland Ankle Instability Tool and that between the smartphone accelerometer and I-Balance were fair. Future studies should investigate the validity of the smartphone accelerometer as a portable medical device for determining ankle instability.

scholarly journals Validation of the pediatric Radboud dysarthria assessment

2021 ◽  
pp. 1-12
Author(s):  
Marieke Ruessink ◽  
Lenie van den Engel-Hoek ◽  
Marjo van Gerven ◽  
Bea Spek ◽  
Bert de Swart ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation ◽  
Self Care ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Activity Level ◽  
Clinical Use ◽  
Spearman Correlation ◽  
Rater Reliability ◽  
Video Recordings

PURPOSE: The Radboud Dysarthria Assessment (RDA) was published in 2014. Adaptation into a pediatric version (p-RDA) was required because of relevant differences between children and adults. The purpose of this study was to assess the feasibility of the p-RDA and to test intra-rater and inter-rater reliability as well as the validity of the two severity scales (function and activity level). METHODS: Video recordings were made of 35 participants with (suspected) dysarthria (age 4 to 17 years) while being assessed using the p-RDA. Intra-rater reliability was assessed by one, and inter-rater reliability by two experiments using the Intraclass Correlation Coefficient (ICC). Validity of the severity scales was tested by correlating the consensus scores with the independently rated scores on four communication scales, three mobility scales, and one self-care scale using Spearman correlation coefficients (r s). RESULTS: The assessment was applicable for 89% of the tested sample, with good intra-rater and inter-rater reliability (ICC = 0.88–0.98 and 0.83–0.93). The p-RDA severity scales (function and activity level) correlated from substantially to strongly with the communication scales (r s = 0.69–0.82 and 0.77–0.92) and self-care scale (r s = 0.76–0.71) and correlated substantially with the mobility scales (r s = 0.49–0.60). CONCLUSION: The feasibility, reliability and validity of the p-RDA are sufficient for clinical use.

scholarly journals Intra- and inter-rater reliability in a comparative study of cross-sectional and spiral computed tomography pelvimetry methods

2019 ◽  
Vol 8 (6) ◽  
pp. 205846011985518 ◽  
Author(s):  
Erika Phexell ◽  
Anna Åkesson ◽  
Marcus Söderberg ◽  
Anetta Bolejko
Keyword(s):  
Computed Tomography ◽  
Radiation Dose ◽  
Confidence Interval ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Measurement Reliability ◽  
Cross Sectional ◽  
Rater Reliability ◽  
Short Spiral

Background Different low-dose computed tomography (CT) pelvimetry methods can be used to evaluate the size of birth canal before delivery. CT pelvimetry might generate an acceptable low fetal radiation dose but its measurement accuracy is unknown. Purpose To investigate intra- and inter-rater measurement reliability of cross-sectional and two spiral CT pelvimetry methods: standard spiral and short spiral. Material and Methods Ten individuals (age ≥60 years, body mass index ≥30 kg/m2) having a CT scan of the abdomen also had CT pelvimetry scans. Three radiologists made independent measurements of each pelvimetry method on two occasions and also in consensus for a reference pelvimetry computed from the standard-dose CT scan of the abdomen. Inter- and intra-rater reliability was analyzed by intraclass correlation coefficient. Results Measurements in the short spiral pelvimetry demonstrated excellent intra- and inter-rater reliability, intraclass correlation coefficient ≥0.93, and good to excellent 95% confidence interval 0.87–0.99. Corresponding results of the standard spiral and cross-sectional pelvimetry showed good to excellent intraclass correlation coefficient ≥0.85 and ≥0.76, and 95% confidence interval was least good and moderate 0.73–0.98 and 0.59–0.97, respectively. Intraclass correlation coefficient between reference pelvimetry and other CT methods showed analogous results. Conclusion The short spiral pelvimetry demonstrated high and best reliability in comparison to other methods. Standard spiral method showed also good measurement reliability but the short spiral pelvimetry generates lower fetal radiation dose. This method might be suitable for measurements at narrow pelvis. Patient acceptance and attitude to CT pelvimetry should be investigated.

scholarly journals Development of the Huddle Observation Tool for structured case management discussions to improve situation awareness on inpatient clinical wards

2017 ◽  
Vol 27 (5) ◽  
pp. 365-372 ◽  
Author(s):  
Julian Edbrooke-Childs ◽  
Jacqueline Hayes ◽  
Evelyn Sharples ◽  
Dawid Gondek ◽  
Emily Stapley ◽  
...  
Keyword(s):  
Case Management ◽  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Situation Awareness ◽  
Assessment Tool ◽  
Intraclass Correlation ◽  
Weighted Kappa ◽  
Observational Assessment ◽  
Rater Reliability ◽  
Team Processes

Background‘Situation Awareness For Everyone’ (SAFE) was a 3-year project which aimed to improve situation awareness in clinical teams in order to detect potential deterioration and other potential risks to children on hospital wards. The key intervention was the ‘huddle’, a structured case management discussion which is central to facilitating situation awareness. This study aimed to develop an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle.MethodsA cross-sectional observational design was used to psychometrically develop the ‘Huddle Observation Tool’ (HOT) over three phases using standardised psychometric methodology. Huddles were observed across four NHS paediatric wards participating in SAFE by five researchers; two wards within specialist children hospitals and two within district general hospitals, with location, number of beds and length of stay considered to make the sample as heterogeneous as possible. Inter-rater reliability was calculated using the weighted kappa and intraclass correlation coefficient.ResultsInter-rater reliability was acceptable for the collaborative culture (weighted kappa=0.32, 95% CI 0.17 to 0.42), environment items (weighted kappa=0.78, 95% CI 0.52 to 1) and total score (intraclass correlation coefficient=0.87, 95% CI 0.68 to 0.95). It was lower for the structure and risk management items, suggesting that these were more variable in how observers rated them. However, agreement on the global score for huddles was acceptable.ConclusionWe developed an observational assessment tool to assess the team processes occurring during huddles, including the effectiveness of the huddle. Future research should examine whether observational evaluations of huddles are associated with other indicators of safety on clinical wards (eg, safety climate and incidents of patient harm), and whether scores on the HOT are associated with improved situation awareness and reductions in deterioration and adverse events in clinical settings, such as inpatient wards.

scholarly journals Further Data on the Reliability of the Mentalization Imbalances Scale and of the Modes of Mentalization Scale

2020 ◽  
Vol 23 (1) ◽  
Author(s):  
Giulia Gagliardini ◽  
Laura Gatti ◽  
Antonello Colli
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Intraclass Correlation ◽  
Test Reliability ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Test Retest Reliability

The aim of this study was to provide data on the Inter-Rater Reliability (IRR) and the test-retest reliability of the Mentalization Imbalances Scale (MIS) and the Modes of Mentalization Scale (MMS) in two different studies. Three junior raters and two senior raters assessed blindly 15 session transcripts of psychotherapy of five patients, using both the MIS and the MMS. The same 15 sessions were rated after the junior raters completed a training at the use of the scales and after on month from the end of the training to assess testretest reliability. Four therapists used the MIS and the MMS to provide different ratings of 22 patients undergoing a psychotherapy in different settings. Intraclass Correlation Coefficient (ICC) values ranged from sufficient to good and increased after the training. Test re-test reliability was sufficient for both scales (Study 1). ICC values ranged from sufficient to good, and were globally higher than the ones found in the first study sample (Study 2). Our results provide support to the inter-rater reliability of the MIS and the MMS.

Inference for variance components in linear mixed-effect models with flexible random effect and error distributions

2020 ◽  
Vol 29 (12) ◽  
pp. 3586-3604
Author(s):  
Tom Chen ◽  
Rui Wang
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Random Effects ◽  
Bootstrap Method ◽  
Intraclass Correlation ◽  
Random Effect ◽  
Correlated Data ◽  
Parameter Distribution ◽  
Mixed Effect ◽  
Error Distributions

In many biomedical investigations, parameters of interest, such as the intraclass correlation coefficient, are functions of higher-order moments reflecting finer distributional characteristics. One popular method to make inference for such parameters is through postulating a parametric random effects model. We relax the standard normality assumptions for both the random effects and errors through the use of the Fleishman distribution, a flexible four-parameter distribution which accounts for the third and fourth cumulants. We propose a Fleishman bootstrap method to construct confidence intervals for correlated data and develop a normality test for the random effect and error distributions. Recognizing that the intraclass correlation coefficient may be heavily influenced by a few extreme observations, we propose a modified, quantile-normalized intraclass correlation coefficient. We evaluate our methods in simulation studies and apply these methods to the Childhood Adenotonsillectomy Trial sleep electroencephalogram data in quantifying wave-frequency correlation among different channels.

Proposing a new short screening test for upper limb apraxia

2021 ◽  
pp. 030802262199856
Author(s):  
Mai Yamada ◽  
Masahiko Koyanagi ◽  
Miyo Kawaguchi ◽  
Yuki Sato ◽  
Mitsuhiro Tsujihata ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Intraclass Correlation Coefficient ◽  
Upper Limb ◽  
Screening Test ◽  
Intraclass Correlation ◽  
Stroke Patients ◽  
Special Equipment ◽  
Rater Reliability ◽  
Verbal Instructions ◽  
Limb Apraxia

Background Apraxia has a major impact on activities of daily living in stroke patients. The proper assessment and treatment of apraxia is important for maintaining a good quality of life. We developed a short evaluation test for upper limb apraxia. Patients and Methods The present Screening Test of Gestures for Stroke consists of 10 items for each verbal instruction and imitation. Each item includes three meaningless gestures, three meaningful gestures and four pantomimes. The Screening Test of Gestures for Stroke is scored based on a 3-point system: 10, 5 or 0 (maximum score: 200). The test took approximately 2–5 min to complete. We recruited 65 patients admitted to our hospital with left hemisphere stroke and 50 healthy subjects. Results The reliability of the Screening Test of Gestures for Stroke was as follows: the intraclass correlation coefficient of intra-rater reliability was 0.93 for both verbal instructions and imitations, and the intraclass correlation coefficient total scores for inter-rater reliability for verbal instructions and for imitations were 0.97 and 0.95, respectively. The alpha coefficient was ≥0.80. Conclusions The Screening Test of Gestures for Stroke is a reliable and valid bedside test that has a short assessment time, does not require special equipment and can evaluate upper limb apraxia in stroke patients from the acute to the chronic phase.

Sign in / Sign up

Export Citation Format

Share Document