Reliability Assessment of an Observation Tool to Measure Praise Characteristics

2020 ◽  
Vol 23 (1) ◽  
pp. 17-29
Author(s):  
Andrew M. Markelz ◽  
Benjamin S. Riden ◽  
Kimberly A. Zoder-Martell ◽  
Joseph E. Miller ◽  
Sarah J. Bolinger
Keyword(s):  
Teacher Development ◽  
Intraclass Correlation ◽  
Social Validity ◽  
Future Research ◽  
Kappa Statistics ◽  
Student Behaviors ◽  
Valuable Contribution ◽  
Kappa Score ◽  
Behavior Specific Praise ◽  
Validity Measures

Supported by decades of research on praise and its effect on student behaviors, we developed the Behavior-Specific Praise–Observation Tool (BSP-OT) to measure characteristics of effective praise. We evaluated interrater reliability of the BSP-OT to measure praise specificity, contingency, and variety using intraclass correlation (ICC) and Cohen’s kappa statistics. In addition, we assessed usefulness and practicality of the tool with social validity measures. Four raters with experience in praise research completed a survey and coded videos ( n = 14) of teachers in authentic situations delivering praise. Overall assessment indicated strong reliability between raters with ICC(2, k) of .80: 95% confidence interval (CI) = [0.77, 0.83], F(269, 19906) = 5.1, p < .001, and mean kappa score of .91. Furthermore, high social validity ratings suggest the BSP-OT is a valuable contribution to the field concerning praise research and teacher development. The process of developing the BSP-OT and study findings are presented, with a discussion of implications and suggestions for future research.

Effects of Tactile Prompting and Self-Monitoring on Teachers’ Use of Behavior-Specific Praise

2019 ◽  
Vol 85 (4) ◽  
pp. 471-489 ◽  
Author(s):  
Andrew M. Markelz ◽  
Jonte C. Taylor ◽  
Tom Kitchen ◽  
Paul J. Riccomini ◽  
Mary Catherine Scheeler ◽  
...  
Keyword(s):  
Students With Disabilities ◽  
Special Education Teachers ◽  
Future Research ◽  
Student Behaviors ◽  
Self Monitoring ◽  
Empirically Supported ◽  
Behavior Specific Praise ◽  
The Impact ◽  
Task Behaviors ◽  
Specific Praise

Effectively managing a classroom is crucial in promoting positive student outcomes. Behavior-specific praise is an empirically supported strategy to reinforce desirable student behaviors. Following a review of the literature, we identified tactile prompting and self-monitoring as effective methods to increase teachers’ use of behavior-specific praise while sustaining intervention long enough until teachers contacted natural maintaining contingencies. We created electronic tactile awareness prompting with self-monitoring (eTAPS) by combining two applications on an Apple Watch. Using a multiple-baseline-across-participants design, this study primarily investigated the effects of eTAPS on special education teachers’ use of behavior-specific praise. Secondarily, this study investigated the impact that behavior-specific praise would have on the on-task behaviors of targeted students with disabilities demonstrating frequent off-task behaviors. Results indicated that eTAPS was effective in increasing and maintaining behavior-specific praise rates. Furthermore, significant increases in student on-task behavior occurred. Implications of results and future research are discussed.

Iterative Development of a Home Review Program to Promote Preschoolers' Vocabulary Skills: Social Validity and Learning Outcomes

2020 ◽  
Vol 51 (2) ◽  
pp. 371-389 ◽  
Author(s):  
Xigrid Soto ◽  
Yagmur Seven ◽  
Meaghan McKenna ◽  
Keri Madsen ◽  
Lindsey Peters-Sanders ◽  
...  
Keyword(s):  
High Stress ◽  
Social Validity ◽  
Home Language ◽  
Single Subject ◽  
Considerable Difficulty ◽  
Home Program ◽  
Iterative Development ◽  
Vocabulary Skills ◽  
Validity Measures ◽  
At Home

Purpose This article describes the iterative development of a home review program designed to augment vocabulary instruction for young children (ages 4 and 5 years) occurring at school through the use of a home review component. Method A pilot study followed by two experiments used adapted alternating treatment designs to compare the learning of academic words taught at school to words taught at school and reviewed at home. At school, children in small groups were taught academic words embedded in prerecorded storybooks for 6 weeks. Children were given materials such as stickers with review prompts (e.g., “Tell me what brave means”) to take home for half the words. Across iterations of the home intervention, the home review component was enhanced by promoting parent engagement and buy-in through in-person training, video modeling, and daily text message reminders. Visual analyses of single-subject graphs, multilevel modeling, and social validity measures were used to evaluate the additive effects and feasibility of the home review component. Results Social validity results informed each iteration of the home program. The effects of the home program across sites were mixed, with only one site showing consistently strong effects. Superior learning was evident in the school + home review condition for families that reviewed words frequently at home. Although the home review program was effective in improving the vocabulary skills of many children, some families had considerable difficulty practicing vocabulary words. Conclusion These studies highlight the importance of using social validity measures to inform iterative development of home interventions that promote feasible strategies for enhancing the home language environment. Further research is needed to identify strategies that stimulate facilitators and overcome barriers to implementation, especially in high-stress homes, to enrich the home language environments of more families.

Linear Mandibular Measurements: Comparison between Orthopantomograms and Lateral Cephalograms

2009 ◽  
Vol 46 (2) ◽  
pp. 147-153 ◽  
Author(s):  
E. M. Ongkosuwito ◽  
M. M. J. Dieleman ◽  
A. M. Kuijpers-Jagtman ◽  
P. G. H. Mulder ◽  
J. W. van Neck
Keyword(s):  
Interobserver Variability ◽  
Intraclass Correlation ◽  
Kappa Statistics ◽  
Magnification Factor ◽  
Linear Measurements ◽  
Lateral Cephalogram ◽  
X Ray ◽  
Lateral Cephalograms ◽  
Left And Right ◽  
Cephalometric Measurements

Objective: To investigate the reliability of length measurements of the mandible by comparing orthopantomograms (OPTs) with lateral cephalograms. Design: Observational study. Setting: OPTs and lateral cephalograms were taken of 20 human dry skulls. Four orthodontists and four maxillofacial surgeons located landmarks on all radiographs using a computer program for cephalometric measurements. Intraobserver and interobserver variability in locating landmarks was assessed, as well as positioning of the skulls prior to radiography between the x-ray assistants. Magnification differences between the left and right side of the mandible on the OPT were determined for five skulls. Kappa statistics were used to calculate the intraclass correlation coefficient for intraobserver and interobserver differences. An F test was used to assess differences between methods and between type of observer. Results: No significant differences were found in the magnification factor of the left and right side of the mandible. Compared with a lateral cephalogram, the OPT had comparable reliability in measuring mandibular distances condylion-gonion, gonion-menton, and condylion-menton. No significant differences were observed between the x-ray assistants in taking the OPTs and lateral cephalograms or in repositioning the skulls. Significant differences were found between orthodontists and maxillofacial surgeons for landmark measurements. Conclusion: An OPT is as reliable as a lateral cephalogram for linear measurements of the mandible (condylion-gonion, gonion-menton, and condylion-menton).

scholarly journals Psychometric Properties of the MyotonPRO in Dementia Patients with Paratonia

Gerontology ◽  
2017 ◽  
Vol 64 (4) ◽  
pp. 401-412 ◽  
Author(s):  
Hans Drenth ◽  
Sytse U. Zuidema ◽  
Wim P. Krijnen ◽  
Ivan Bautmans ◽  
Cees van der Schans ◽  
...  
Keyword(s):  
Psychometric Properties ◽  
Correlation Coefficient ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Functional Mobility ◽  
Future Research ◽  
Minimal Detectable Change ◽  
Muscle Properties ◽  
Dementia Patients ◽  
Longitudinal Outcome

Background: Paratonia is a distinctive form of hypertonia, causing loss of functional mobility in early stages of dementia to severe high muscle tone and pain in the late stages. For assessing and evaluating therapeutic interventions, objective instruments are required. Objective: Determine the psychometric properties of the MyotonPRO, a portable device that objectively measures muscle properties, in dementia patients with paratonia. Methods: Muscle properties were assessed with the MyotonPRO by 2 assessors within one session and repeated by the main researcher after 30 min and again after 6 months. Receiver operating characteristic curves were constructed for all MyotonPRO outcomes to discriminate between participants with (n = 70) and without paratonia (n = 82). In the participants with paratonia, correlation coefficients were established between the MyotonPRO outcomes and the Modified Ashworth Scale for paratonia (MAS-P) and muscle palpation. In participants with paratonia, reliability (intraclass correlation coefficient) and agreement values (standard error of measurement and minimal detectable change) were established. Longitudinal outcome from participants with paratonia throughout the study (n = 48) was used to establish the sensitivity for change (correlation coefficient) and responsiveness (minimal clinical important difference). Results: Included were 152 participants with dementia (mean [standard deviation] age of 83.5 [98.2]). The area under the curve ranged from 0.60 to 0.67 indicating the MyotonPRO is able to differentiate between participants with and without paratonia. The MyotonPRO explained 10-18% of the MAS-P score and 8-14% of the palpation score. Interclass correlation coefficients for interrater reliability ranged from 0.57 to 0.75 and from 0.54 to 0.71 for intrarater. The best agreement values were found for tone, elasticity, and stiffness. The change between baseline and 6 months in the MyotonPRO outcomes explained 8-13% of the change in the MAS-P scores. The minimal clinically important difference values were all smaller than the measurement error. Conclusion: The MyotonPRO is potentially applicable for cross-sectional studies between groups of paratonia patients and appears less suitable to measure intraindividual changes in paratonia. Because of the inherent variability in movement resistance in paratonia, the outcomes from the MyotonPRO should be interpreted with care; therefore, future research should focus on additional guidelines to increase the clinical interpretation and improving reproducibility.

How to Measure Client Satisfaction With Stop Smoking Services: A Pilot Project in the UK National Health Service

2009 ◽  
Vol 4 (1) ◽  
pp. 52-58 ◽  
Author(s):  
Sylvia May ◽  
Andy McEwen ◽  
Helen Arnoldi ◽  
Linda Bauld ◽  
Janet Ferguson ◽  
...  
Keyword(s):  
National Health ◽  
Client Satisfaction ◽  
Survey Methodology ◽  
Smoking Status ◽  
Intraclass Correlation ◽  
Face Validity ◽  
Future Research ◽  
Stop Smoking ◽  
Stop Smoking Services ◽  
Service To Others

AbstractThis pilot study aimed to develop a tool and methodology for measuring client satisfaction in UK National Health (NHS) Stop Smoking Services (SSS). A brief postcard questionnaire (measuring overall satisfaction with the service, willingness to recommend the service to others and smoking status) and a complete questionnaire (with 20 additional items measuring satisfaction with specific elements of the service) were developed. An NHS SSS mailed the postcard to 298 clients who had set a quit date in the previous quarter, they mailed the complete questionnaire to a subsample of 99 clients. Overall 34% (100/298) of those surveyed responded: 30% (90/298) for the card and 25% (25/99) for the questionnaire (15 people responded to both). Intraclass correlation coefficients (ICC) were found to be acceptable for both the overall service satisfaction item (ICC value = .43, p = .05) and the item regarding recommending the service to others (ICC-value = .83, p < .001). Hence the tool had reliability and at least face validity and the survey methodology proved practicable. The small modifications made to service delivery and the need for future research are discussed.

Comparing Counts of Park Users With a Wearable Video Device and an Unmanned Aerial System

2021 ◽  
pp. 1-8
Author(s):  
Richard R. Suminski ◽  
Gregory M. Dominick ◽  
Matthew Saponaro
Keyword(s):  
Repeated Measures ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Future Research ◽  
Unmanned Aerial Systems ◽  
Park Use ◽  
Intraclass Correlation Coefficients ◽  
Potential Benefits ◽  
Absolute Agreement ◽  
Aerial Systems

Evidence suggests that video captured with a wearable video device (WVD) may augment or supplant traditional methods for assessing park use. Unmanned aerial systems (UASs) are used to assess human activity, but research employing them for park assessments is sparse. Therefore, this study compared park user counts between a WVD and UAS. A diverse set of 33 amenities (e.g., playground) in three parks were videoed simultaneously by one researcher wearing a WVD and another operating the UAS. Assessments were done at 12 p.m. and 7 p.m. on weekends, with one park evaluated on two occasions 7 days apart. Two investigators independently reviewed videos and reached consensus on the counts of individuals at each amenity. Intraclass correlation coefficients (ICCs) were used to determine intra- and interrater reliabilities. A total of 404 (M = 4.7; SD = 9.6) and 389 (M = 4.5; SD = 9.0) individuals were counted in the UAS and WVD videos, respectively. Absolute agreement was 86% (74/86) and 100% when no individuals were using the amenity. Whether using all 86 videos or only videos having people (48 videos), ICCs indicated excellent reliability (ICC = .99; p < .001). The totals seen for the repeated measures were UAS = 146 and WVD = 136 for Day 1 and UAS = 169 and WVD = 161 for Day 2. Intrarater reliability was excellent for the UAS (ICC = .92; p < .001) and good for the WVD (ICC = .89; p < .001). Disagreement was mainly due to obstructions—people behind or under structures. This study provides support for the use of UASs for counting park users and future research examining the potential benefits of video analysis for assessing park use.

scholarly journals Standardising the measurement of physical activity in people receiving haemodialysis: considerations for research and practice

2019 ◽  
Vol 20 (1) ◽  
Author(s):  
Hannah M. L. Young ◽  
Mark W. Orme ◽  
Yan Song ◽  
Maurice Dungey ◽  
James O. Burton ◽  
...  
Keyword(s):  
Physical Activity ◽  
Sample Size ◽  
Repeated Measures ◽  
A Priori ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Future Research ◽  
Wear Time ◽  
Minimum Number ◽  
The Uk

Abstract Background Physical activity (PA) is exceptionally low amongst the haemodialysis (HD) population, and physical inactivity is a powerful predictor of mortality, making it a prime focus for intervention. Objective measurement of PA using accelerometers is increasing, but standard reporting guidelines essential to effectively evaluate, compare and synthesise the effects of PA interventions are lacking. This study aims to (i) determine the measurement and processing guidance required to ensure representative PA data amongst a diverse HD population, and; (ii) to assess adherence to PA monitor wear amongst HD patients. Methods Clinically stable HD patients from the UK and China wore a SenseWear Armband accelerometer for 7 days. Step count between days (HD, Weekday, Weekend) were compared using repeated measures ANCOVA. Intraclass correlation coefficients (ICCs) determined reliability (≥0.80 acceptable). Spearman-Brown prophecy formula, in conjunction with a priori ≥  80% sample size retention, identified the minimum number of days required for representative PA data. Results Seventy-seven patients (64% men, mean ± SD age 56 ± 14 years, median (interquartile range) time on HD 40 (19–72) months, 40% Chinese, 60% British) participated. Participants took fewer steps on HD days compared with non-HD weekdays and weekend days (3402 [95% CI 2665–4140], 4914 [95% CI 3940–5887], 4633 [95% CI 3558–5707] steps/day, respectively, p < 0.001). PA on HD days were less variable than non-HD days, (ICC 0.723–0.839 versus 0.559–0.611) with ≥ 1 HD day and ≥  3 non-HD days required to provide representative data. Using these criteria, the most stringent wear-time retaining ≥ 80% of the sample was ≥7 h. Conclusions At group level, a wear-time of ≥7 h on ≥1HD day and ≥ 3 non-HD days is required to provide reliable PA data whilst retaining an acceptable sample size. PA is low across both HD and non- HD days and future research should focus on interventions designed to increase physical activity in both the intra and interdialytic period.

scholarly journals All insects are equal, but some insects are more equal than others

2018 ◽  
Vol 120 (4) ◽  
pp. 852-863 ◽  
Author(s):  
Arnout R.H. Fischer ◽  
L.P.A. (Bea) Steenbekkers
Keyword(s):  
Design Methodology ◽  
Prior Experience ◽  
Future Research ◽  
Valuable Contribution ◽  
Short Term ◽  
Content Type ◽  
Quasi Experimental ◽  
Insects As Food ◽  
Practical Implications

Purpose Lack of acceptance of insects as food is considered a barrier against societal adoption of the potentially valuable contribution of insects to human foods. An underlying barrier may be that insects are lumped together as one group, while consumers typically try specific insects. The purpose of this paper is to investigate the ways in which Dutch consumers, with and without insect tasting experience, are more or less willing to eat different insects. Design/methodology/approach In a quasi-experimental study (n=140), the participants with and without prior experience in eating insects were asked to give their willingness to eat a range of insects, and their attitudes and disgust towards eating insects. Findings Insects promoted in the market were more preferred than the less marketed insects, and a subgroup of preferred insects for participants with experience in eating insects was formed. Research limitations/implications Although well-known insects were more preferred, general willingness to eat remained low for all participants. The results indicate that in future research on insects as food the specific insects used should be taken into account. Practical implications Continued promotion of specific, carefully targeted, insects may not lead to short-term uptake of insects as food, but may contribute to willingness to eat insects as human food in the long term. Originality/value The paper shows substantial differences between consumers who have and who have not previously tasted insects, with higher acceptance of people with experience in tasting insects for the specific insects that are frequently promoted beyond their generally more positive attitude towards eating insects.

scholarly journals Test UHCJ20m—Measurement Procedure Standardization and Metric Characteristics Determination

Sensors ◽  
2020 ◽  
Vol 20 (14) ◽  
pp. 3971
Author(s):  
Aleš Dolenec ◽  
Ivan Milinović ◽  
Vesna Babić ◽  
Dražan Dizdar
Keyword(s):  
Research Study ◽  
High Reliability ◽  
Intraclass Correlation ◽  
Measurement Procedure ◽  
Future Research ◽  
Maximum Speed ◽  
Sports Events ◽  
Jump Test ◽  
The Subject ◽  
Very High

The purpose of the research study was to standardize the measurement procedure and determine the reliability, homogeneity, and sensitivity of a 20 m unilateral horizontal cyclic jump test (UHCJ20m) whose intentional (assumed) measurement aim is the lower extremities’ explosive strength. The subject sample consisted of 31 students from Zagreb University (20.68 ± 1.96 years of age, height 185.16 ± 7.19 cm, body mass 79.48 ± 9.23 kg) actively involved in various sports events. The UHCJ20m test was performed three times using a dominant (take-off) leg with an active rest of 15 min between the repetitions. The results showed that the UHCJ20m test had satisfactory sensitivity and a very high reliability: Cronbach α = 0.95, intraclass correlation coefficient (ICC) = 0.94 and homogeneity average intertrial correlation (AVR) = 0.88. Future research studies should be aimed at determining the metric characteristics of the UHCJ20m test with a population of athletes in sports characterized by start acceleration and maximum speed running.

scholarly journals Gait Asymmetry Post-Stroke: Determining Valid and Reliable Methods Using a Single Accelerometer Located on the Trunk

Sensors ◽  
2019 ◽  
Vol 20 (1) ◽  
pp. 37 ◽  
Author(s):  
Christopher Buckley ◽  
M. Encarna Micó-Amigo ◽  
Michael Dunne-Willows ◽  
Alan Godfrey ◽  
Aodhán Hickey ◽  
...  
Keyword(s):  
Concurrent Validity ◽  
Intraclass Correlation ◽  
Future Research ◽  
Validity And Reliability ◽  
Gait Symmetry ◽  
Retest Reliability ◽  
Post Stroke ◽  
Gait Asymmetry ◽  
Test Retest Reliability ◽  
Cardinal Symptom

Asymmetry is a cardinal symptom of gait post-stroke that is targeted during rehabilitation. Technological developments have allowed accelerometers to be a feasible tool to provide digital gait variables. Many acceleration-derived variables are proposed to measure gait asymmetry. Despite a need for accurate calculation, no consensus exists for what is the most valid and reliable variable. Using an instrumented walkway (GaitRite) as the reference standard, this study compared the validity and reliability of multiple acceleration-derived asymmetry variables. Twenty-five post-stroke participants performed repeated walks over GaitRite whilst wearing a tri-axial accelerometer (Axivity AX3) on their lower back, on two occasions, one week apart. Harmonic ratio, autocorrelation, gait symmetry index, phase plots, acceleration, and jerk root mean square were calculated from the acceleration signals. Test–retest reliability was calculated, and concurrent validity was estimated by comparison with GaitRite. The strongest concurrent validity was obtained from step regularity from the vertical signal, which also recorded excellent test–retest reliability (Spearman’s rank correlation coefficients (rho) = 0.87 and Intraclass correlation coefficient (ICC21) = 0.98, respectively). Future research should test the responsiveness of this and other step asymmetry variables to quantify change during recovery and the effect of rehabilitative interventions for consideration as digital biomarkers to quantify gait asymmetry.

Sign in / Sign up

Export Citation Format

Share Document