scholarly journals A Mobile App to Rapidly Appraise the In-Store Food Environment: Reliability, Utility, and Construct Validity Study

10.2196/16971 ◽  
2020 ◽  
Vol 8 (7) ◽  
pp. e16971 ◽  
Author(s):  
Emma Joy McMahon ◽  
Rachael Jaenke ◽  
Julie Brimblecombe
Keyword(s):  
Construct Validity ◽  
Internal Consistency ◽  
Food Environment ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
End Users ◽  
Mobile App ◽  
Convenience Stores ◽  
Significant Difference ◽  
Store Environment

Background Consumer food environments are increasingly being recognized as influential determinants of food purchasing and subsequent intake and health. We developed a tool to enable efficient, but relatively comprehensive, appraisal of the in-store food environment. The Store Scout mobile app facilitates the evaluation of product (availability and range), placement (visibility, accessibility, proximity to high-traffic areas, and location relative to other products), price (price promotion), and promotion (displays and advertising) across 7 categories of food products, with appraisal given immediately as scores (0-100, where a higher score is more in line with best practice). Primary end users are public health nutritionists and nutritionists employed by store organizations; however, store managers and staff are also potential end users. Objective This study aims to evaluate the reliability (interrater reliability and internal consistency), utility (distribution of scores), and construct validity (score by store type) of measurements using the Store Scout mobile app. Methods The Store Scout mobile app was used independently by 2 surveyors to evaluate the store environment in 54 stores: 34 metropolitan stores (9 small and 11 large supermarkets, 10 convenience stores, and 4 petrol stations) in Brisbane, Australia, and 20 remote stores (19 small supermarkets and 1 petrol station) in Indigenous Australian communities in Northern Australia. The agreement between surveyors in the overall and category scores was evaluated using intraclass correlation coefficients (ICCs). Interrater reliability of measurement items was assessed using percentage agreement and the Gwet agreement coefficient (AC). Internal consistency was assessed by comparing the responses of items measuring similar aspects of the store environment. We examined the distribution of score values using boxplots and differences by store type using the Kruskal-Wallis test. Results The median difference in the overall score between surveyors was 4.4 (range 0.0-11.1), with an ICC of 0.954 (95% CI 0.914-0.975). Most measurement items had very good (n=74/196, 37.8%) or good (n=81/196, 41.3%) interrater reliability using the Gwet AC. A minimal inconsistency of measurement was found. Overall scores ranged from 19.2 to 81.6. There was a significant difference in score by store type (P<.001). Large Brisbane supermarkets scored highest (median 77.4, range 53.2-81.6), whereas small Brisbane supermarkets (median 63.9, range 41.0-71.3) and small remote supermarkets (median 63.8, range 56.5-74.9) scored significantly higher than Brisbane petrol stations (median 33.1, range 19.2-37.8) and convenience stores (median 39.0, range 22.4-63.8). Conclusions These findings suggest good reliability and internal consistency of food environment measurements using the Store Scout mobile app. We identified specific aspects that can be improved to further increase the reliability of this tool. We found a good distribution of score values and evidence that scoring could capture differences by store type in line with previous evidence, which gives an indication of construct validity. The Store Scout mobile app shows promise in its capability to measure and track the health-enabling characteristics of store environments.

scholarly journals A Mobile App to Rapidly Appraise the In-Store Food Environment: Reliability, Utility, and Construct Validity Study (Preprint)

2019 ◽  
Author(s):  
Emma Joy McMahon ◽  
Rachael Jaenke ◽  
Julie Brimblecombe
Keyword(s):  
Construct Validity ◽  
Internal Consistency ◽  
Food Environment ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
End Users ◽  
Mobile App ◽  
Convenience Stores ◽  
Significant Difference ◽  
Store Environment

BACKGROUND Consumer food environments are increasingly being recognized as influential determinants of food purchasing and subsequent intake and health. We developed a tool to enable efficient, but relatively comprehensive, appraisal of the in-store food environment. The Store Scout mobile app facilitates the evaluation of product (availability and range), placement (visibility, accessibility, proximity to high-traffic areas, and location relative to other products), price (price promotion), and promotion (displays and advertising) across 7 categories of food products, with appraisal given immediately as scores (0-100, where a higher score is more in line with best practice). Primary end users are public health nutritionists and nutritionists employed by store organizations; however, store managers and staff are also potential end users. OBJECTIVE This study aims to evaluate the reliability (interrater reliability and internal consistency), utility (distribution of scores), and construct validity (score by store type) of measurements using the Store Scout mobile app. METHODS The Store Scout mobile app was used independently by 2 surveyors to evaluate the store environment in 54 stores: 34 metropolitan stores (9 small and 11 large supermarkets, 10 convenience stores, and 4 petrol stations) in Brisbane, Australia, and 20 remote stores (19 small supermarkets and 1 petrol station) in Indigenous Australian communities in Northern Australia. The agreement between surveyors in the overall and category scores was evaluated using intraclass correlation coefficients (ICCs). Interrater reliability of measurement items was assessed using percentage agreement and the Gwet agreement coefficient (AC). Internal consistency was assessed by comparing the responses of items measuring similar aspects of the store environment. We examined the distribution of score values using boxplots and differences by store type using the Kruskal-Wallis test. RESULTS The median difference in the overall score between surveyors was 4.4 (range 0.0-11.1), with an ICC of 0.954 (95% CI 0.914-0.975). Most measurement items had very good (n=74/196, 37.8%) or good (n=81/196, 41.3%) interrater reliability using the Gwet AC. A minimal inconsistency of measurement was found. Overall scores ranged from 19.2 to 81.6. There was a significant difference in score by store type (<i>P</i>&lt;.001). Large Brisbane supermarkets scored highest (median 77.4, range 53.2-81.6), whereas small Brisbane supermarkets (median 63.9, range 41.0-71.3) and small remote supermarkets (median 63.8, range 56.5-74.9) scored significantly higher than Brisbane petrol stations (median 33.1, range 19.2-37.8) and convenience stores (median 39.0, range 22.4-63.8). CONCLUSIONS These findings suggest good reliability and internal consistency of food environment measurements using the Store Scout mobile app. We identified specific aspects that can be improved to further increase the reliability of this tool. We found a good distribution of score values and evidence that scoring could capture differences by store type in line with previous evidence, which gives an indication of construct validity. The Store Scout mobile app shows promise in its capability to measure and track the health-enabling characteristics of store environments.

scholarly journals Psychometric analysis of the Brazilian-version Kidscreen-27 questionnaire

2021 ◽  
Vol 19 (1) ◽  
Author(s):  
Pablo Magno da Silveira ◽  
Alexsandra da Silva Bandeira ◽  
Marcus Vinicius Veber Lopes ◽  
Adriano Ferreti Borgatto ◽  
Kelly Samara da Silva
Keyword(s):  
Public Schools ◽  
Construct Validity ◽  
Internal Consistency ◽  
Intraclass Correlation ◽  
Well Being ◽  
Discriminatory Power ◽  
Brazilian Version ◽  
Related Quality ◽  
Health Related ◽  
Delta G

Abstract Background The objective of this study was to verify the reliability, discriminatory power and construct validity of the Kidscreen-27 questionnaire in Brazilian adolescents. Methods Adolescents that participated of the pilot study (210 adolescents; 52.9% boys; 13.7 years old) and of the baseline (816 participants; 52.7% girls; 13.1 years old) of the Movimente Project in 2016/2017 composed the sample of the present study. This project was carried out in six public schools in the city of Florianópolis, Santa Catarina, Brazil. Test–retest reproducibility was assessed by the intraclass correlation coefficient and Gwet coefficient; internal consistency through McDonald's Omega; Hankins' Delta G coefficient verified the scale's discriminatory power and; confirmatory factor analysis to assess construct validity. Results Reproducibility values ranged from 0.71 to 0.78 for the dimensions (ICC), and ranged from 0.60 to 0.83 for the items (Gwet). McDonald's Ômega (0.82–0.91) for internal consistency measures. Discriminatory power ranging from 0.94 for the dimension Social Support and Friends to 0.98 for Psychological Well-Being. The factorial loads were > 0.40, except for item 19 (0.36). The fit quality indicators of the model were adequate (X2[df] = 1022.89 [311], p < 0.001; RMSEA = 0.053 (0.049–0.087); CFI = 0.988; TLI = 0.987), confirming the five-factor structure originally proposed. Conclusions The Brazilian-version Kidscreen-27 achieved good levels of reproducibility, internal consistency, discriminatory power and construct validity. Its use is adequate to measure the health-related quality of life of adolescents in the Brazilian context.

scholarly journals Reliability of Physiological Responses Induced by Basic Emotions: A Pilot Study

2019 ◽  
Vol 38 (1) ◽  
Author(s):  
Eun-Hye Jang ◽  
Sangwon Byun ◽  
Mi-Sook Park ◽  
Jin-Hun Sohn
Keyword(s):  
Internal Consistency ◽  
Interrater Reliability ◽  
Physiological Responses ◽  
Skin Conductance Level ◽  
Intraclass Correlation ◽  
Repeated Measurements ◽  
Physiological Signals ◽  
Basic Emotions ◽  
Reliability Indices ◽  
Film Clips

Abstract Background Although emotion-specific autonomic responses based on the discrete theory of emotion have been widely studied, studies on the reliability of physiological responses to emotional stimuli are limited. In this study, we aimed to assess the reliability of physiological changes induced by the six basic emotions (happiness, sadness, anger, fear, disgust, and surprise) that were measured during 10 weekly repeated experiments. Methods Twelve college students participated, and in each experiment, physiological signals were collected before and while participants were watching emotion-provoking film clips. Additionally, the participants self-evaluated the emotions that they experienced during the film presentation at the end of each emotional stimulus. To avoid adaptation of participants to identical stimuli during repeated measurements, we used 10 different film clips for each emotion, and thus a total of 60 film clips over 10 weeks were used. Physiological features, such as skin conductance level (SCL), fingertip temperature (FT), heart rate (HR), and blood volume pulse (BVP), were extracted from the physiological signals. Two reliability indices, Cronbach’s alpha and intraclass correlation coefficient, were calculated from the physiological features to assess internal consistency and interrater reliability, respectively. Results We found that SCL, HR, and BVP measured during the emotion-provoking phase over the 10 weekly sessions were more reliable than those assessed at baseline. Furthermore, SCL, HR, and BVP from the emotion-provoking phase exhibited excellent internal consistency and interrater reliability. Conclusions Our findings suggest that these features can be used as reliable physiological indices in emotion studies. The results also support the significance of physiological signals as meaningful indicators for emotion recognition in HCI (human computer interface) area.

scholarly journals Validation of an Evidence-Based Medicine Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET)

2013 ◽  
Vol 5 (2) ◽  
pp. 252-256 ◽  
Author(s):  
Hans B. Kersten ◽  
John G. Frohna ◽  
Erin L. Giudice
Keyword(s):  
Internal Consistency ◽  
Evidence Based Medicine ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Evidence Based ◽  
Evaluation Tool ◽  
Intraclass Correlation Coefficients ◽  
Excellent Internal Consistency ◽  
Based Medicine

Abstract Background Competence in evidence-based medicine (EBM) is an important clinical skill. Pediatrics residents are expected to acquire competence in EBM during their education, yet few validated tools exist to assess residents' EBM skills. Objective We sought to develop a reliable tool to evaluate residents' EBM skills in the critical appraisal of a research article, the development of a written EBM critically appraised topic (CAT) synopsis, and a presentation of the findings to colleagues. Methods Instrument development used a modified Delphi technique. We defined the skills to be assessed while reviewing (1) a written CAT synopsis and (2) a resident's EBM presentation. We defined skill levels for each item using the Dreyfus and Dreyfus model of skill development and created behavioral anchors using a frame-of-reference training technique to describe performance for each skill level. We evaluated the assessment instrument's psychometric properties, including internal consistency and interrater reliability. Results The EBM Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET) is composed of 14 items that assess residents' EBM and global presentation skills. Resident presentations (N  =  27) and the corresponding written CAT synopses were evaluated using the EBM C-PET. The EBM C-PET had excellent internal consistency (Cronbach α  =  0.94). Intraclass correlation coefficients were used to assess interrater reliability. Intraclass correlation coefficients for individual items ranged from 0.31 to 0.74; the average intraclass correlation coefficients for the 14 items was 0.67. Conclusions We identified essential components of an assessment tool for an EBM CAT synopsis and presentation with excellent internal consistency and a good level of interrater reliability across 3 different institutions. The EBM C-PET is a reliable tool to document resident competence in higher-level EBM skills.

scholarly journals The psychometric properties of the respectful maternity care (RMC) for an Iranian population

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Khadije Hajizadeh ◽  
Mohammad Asghari Jafarabadi ◽  
Maryam Vaezi ◽  
Shahla Meedya ◽  
Sakineh Mohammad-Alizadeh-Charandabi ◽  
...  
Keyword(s):  
Construct Validity ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Maternity Care ◽  
Intraclass Correlation ◽  
Simple Random Sampling ◽  
Factor Loading ◽  
Respectful Maternity Care ◽  
Cronbach’S Alpha ◽  
Cronbach's Alpha

Abstract Background The absence of Respectful Maternity Care (RMC) deters mothers from seeking maternity care services. Given the importance of RMC and the lack of a standard tool for its assessment in Iran, the present study was conducted to translate and assess the psychometric properties of the RMC questionnaire in Iranian women. Methods Forward-backward method was used for translating the questionnaire from English into Persian. A total of 265 postpartum women entered the study by simple random sampling from public and private hospitals in Tabriz, Iran. The validity of the questionnaire was confirmed through the face, content and construct validity. Construct validity was assessed through exploratory and confirmatory factor analyses. The internal consistency and test-retest reliability were used to confirm the reliability of the questionnaire. Internal consistency was examined by measuring the Cronbach’s alpha in a sample of 20 mothers, and test-retest stability by calculating the Intraclass Correlation Coefficient (ICC) in the same group of mothers, who had completed the questionnaire twice with a two-week interval. Results The exploratory factor analysis led to the extraction of one factor. Item 12 was eliminated due to its low factor loading. X2/df was less than 5, and RMSEA was less than 0.08, which confirms the validity of this model. The Cronbach’s alpha coefficient was obtained as 0.93 and ICC (with 95% confidence interval) as 0.98 (0.96 to 0.99). Conclusion The results of the study demonstrated that the Iranian RMC scale can be used as a valid and reliable instrument to assess RMC in Iran.

scholarly journals Psychometric of the Curiosity and Exploration Inventory-II in Indonesia

2020 ◽  
Vol 9 (3) ◽  
Author(s):  
Anggi Setyowati ◽  
Min-Huey Chung ◽  
Ah. Yusuf ◽  
Setya Haksama
Keyword(s):  
Factor Analysis ◽  
Construct Validity ◽  
Internal Consistency ◽  
Structural Model ◽  
Intraclass Correlation ◽  
Cronbach’S Alpha ◽  
Retest Reliability ◽  
Cronbach's Alpha ◽  
Test Retest Reliability ◽  
Exploration Inventory

Background: Curiosity is a personality characteristic, which fits with wellbeing and positive functioning. The objective of this study was to assess the construct validity of the Curiosity and Exploration Inventory II (CEI-II) in Indonesia.Design and Methods: The study included 256 undergraduate students who lived in Indonesia, mean age 19.8 years old. The CEI-II measures stretching and embracing using 11 items. The English version of CEI-II was translated into Bahasa. The Cronbach’s alpha coefficient and intraclass correlation coefficient (ICC) were addressed to examine internal consistency reliability and the test-retest reliability. To evaluate construct validity, exploratory factor analysis (EFA) was used to assess factor structure and confirmatory factor analysis (CFA) was used to evaluate the structural model fit of the CEI-II Indonesia version.Results: The study showed Cronbach’s alpha for the internal consistency of the overall CEI-II Indonesia version was 0.77. The ICC for the test-retest reliability ranged between 0.753-0.829. EFA showed adequate with the Kaiser-Meyer-Olkin value of 0.86 and the Bartlett’s test of sphericity was statistically significant. CFA tested the second-order model with two-order factors and showed a model fit.Conclusions: The CEI-II Indonesia version indicated acceptable construct validity to evaluate curiosity in Indonesia.

scholarly journals Assessing Balance Function in Patients With Total Knee Arthroplasty

2015 ◽  
Vol 95 (10) ◽  
pp. 1397-1407 ◽  
Author(s):  
Andy C.M. Chan ◽  
Marco Y.C. Pang
Keyword(s):  
Total Knee Arthroplasty ◽  
Internal Consistency ◽  
Knee Arthroplasty ◽  
Convergent Validity ◽  
Interrater Reliability ◽  
Intraclass Correlation ◽  
Cronbach Alpha ◽  
Ceiling Effects ◽  
Total Knee ◽  
Abc Scale

BackgroundThe Balance Evaluation Systems Test (BESTest) is a relatively new balance assessment tool. Recently, the Mini-BESTest and the Brief-BESTest, which are shortened versions of the BESTest, were developed.ObjectiveThe purpose of this study was to estimate interrater and intrarater-interoccasion reliability, internal consistency, concurrent and convergent validity, and floor and ceiling effects of the 3 BESTests and other related measures, namely, the Berg Balance Scale (BBS), Functional Gait Assessment (FGA), and Activities-specific Balance Confidence (ABC) Scale, among patients with total knee arthroplasty (TKA).DesignThis was an observational measurement study.MethodsTo establish interrater reliability, the 3 BESTests were administered by 3 independent raters to 25 participants with TKA. Intrarater-interoccasion reliability was evaluated in 46 participants with TKA (including the 25 individuals who participated in the interrater reliability experiments) by repeating the 3 BESTests, BBS, and FGA within 1 week by the same rater. Internal consistency of each test also was assessed with Cronbach alpha. Validity was assessed in another 46 patients with TKA by correlating the 3 BESTests with BBS, FGA, and ABC. The floor and ceiling effects also were examined.ResultsThe 3 BESTests demonstrated excellent interrater reliability (intraclass correlation coefficient [ICC] [2,1]=.96–.99), intrarater-interoccasion reliability (ICC [2,1]=.92–.96), and internal consistency (Cronbach alpha=.96–.98). These values were comparable to those for the BBS and FGA. The 3 BESTests also showed moderate-to-strong correlations with the BBS, FGA, and ABC (r=.35–.81), thus demonstrating good concurrent and convergent validity. No significant floor and ceiling effects were observed, except for the BBS.LimitationsThe results are generalizable only to patients with TKA due to end-stage knee osteoarthritis.ConclusionsThe 3 BESTests have good reliability and validity for evaluating balance in people with TKA. The Brief-BESTest is the least time-consuming and may be more useful clinically.

scholarly journals Validitas dan Reliabilitas Ujian SOCA (Students Oral Case Analysis): Studi di Salah Satu Fakultas Kedokteran di Indonesia

2016 ◽  
Vol 5 (3) ◽  
pp. 148
Author(s):  
Rizka Aries Putranti ◽  
Ova Emilia ◽  
Efrayim Suryadi
Keyword(s):  
Construct Validity ◽  
Internal Consistency ◽  
Content Validity ◽  
Cognitive Complexity ◽  
Case Analysis ◽  
Medical Faculty ◽  
Validity And Reliability ◽  
Rater Reliability ◽  
Significant Difference ◽  
Kappa Value

Background: Medical faculty has to make sure that the students meet the minimal competence needed using apropriate exam. While the exam itself should facilitate students to learn. Oral examination has known for its ability to facilitate students learn but low in validity and reliability. Medical faculty of Lampung University (FK Unila) apply the student oral case analysis (SOCA) exam as one of block assessment component, as with MCQ, tutorial, and laboratory exam. This study aimed to evaluate validity and reliability of SOCA examination at FK UnilaMethod: Video of 65 students doing SOCA examination and 28 question rubrics had taken when odd semester exam year 2014-2015 has been carying out at FK Unila. Video and question rubrics were assessed by 5 panelis and analysed using Lawshe's content validity ratio (CVR) to determinate its content validity. Students performance on the video were re-assessed by another assessor to see inter-rater reliability, than analysed using kappa Cohen. Two expert in medical education assessed the cognitive comlpexity of the question rubrics. Data of SOCA's student's mark from year II, III, and IV were analysed for construct valdity and internal consistency.Results: 93,7% of the overall question in 65 video were valid (CVR>99%) and 71,8% question number in 28 question rubrics also valid according to 5 panelis. SOCA cognitive complexity were at level of analyse, know how and 4a. Inter-rater reliability analysis showed 0,549 (moderate agreement) kappa value. Mann Whitney analysis for construct validity showed no significant difference of all year. Cronbach alpha analysis showed internal consistency at the point 0,575.Conclusion: FK Unila's SOCA of odd semester examination year 2014-2015 has sufficient content validity, sufficient cognitive complexity and sufficent inter-rater reliability but lack in construct validity and internal consistency. Keywords: SOCA, validity, reliability

scholarly journals Application of a Simulation-Based Interprofessional Teamwork Assessment Tool (SITAT) to Individual Student Performance in a Team-Based Simulation

2021 ◽  
Vol 8 ◽  
pp. 238212052110424
Author(s):  
Brittany J Daulton ◽  
Laura Romito ◽  
Zach Weber ◽  
Jennifer Burba ◽  
Rami A Ahmed
Keyword(s):  
Student Performance ◽  
Internal Consistency ◽  
Interprofessional Education ◽  
Interrater Reliability ◽  
Assessment Tool ◽  
Individual Performance ◽  
Individual Student ◽  
Interprofessional Teamwork ◽  
Significant Difference ◽  
Simulation Based

There are a very limited number of instruments to assess individual performance in simulation-based interprofessional education (IPE). The purpose of this study was to apply the Simulation-Based Interprofessional Teamwork Assessment Tool (SITAT) to the individualized assessment of medicine, pharmacy, and nursing students (N = 94) in a team-based IPE simulation, as well as to explore potential differences between disciplines, and calculate reliability estimates for utilization of the tool. Results of an analysis of variance provided evidence that there was no statistically significant difference among professions on overall competency ( F(2, 91)  =  0.756, P  = .472). The competency reports for nursing ( M = 3.06, SD = 0.45), medicine ( M = 3.19, SD = 0.42), and pharmacy ( M = 3.08, SD = 0.49) students were comparable across professions. Cronbach's alpha provided a reliability estimate of the tool, with evidence of high internal consistency ( α = .92). The interrater reliability of the SITAT was also investigated. There was moderate absolute agreement across the 3 faculty raters using the 2-way mixed model design and “average” unit (kappa = 0.536, P = .000, 95% CI [0.34, 0.68]). The novel SITAT demonstrates internal consistency and interrater reliability when used for evaluation of individual performance during IPE simulation. The SITAT provides value in the education and evaluation of individual students engaged in IPE curriculum.

Sign in / Sign up

Export Citation Format

Share Document