Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

Background In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs). Methods Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity). Results The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = − 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer’s unit, suggesting face validity. Conclusions Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach. Trial registration Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844.

The Development of Indonesian Accounting Teacher Professional Identity Measurement: An Exploratory Factor Analysis

<p style="text-align: justify;">Measuring accounting teachers’ professional identity is significant to do as an alternative to measure the professionalism of accounting teachers in Indonesia based on their professional identity. This research was conducted in two stages of exploratory factor analysis involving 150 accounting teachers as sample in each stage. The data were collected in collaboration with an accounting teacher organization, comprising the Accounting Subject Teacher Deliberation (MGMP) in Central Java through a questionnaire. Data analysis was divided into several steps including face validity and content validity, inter-item correlation matrix, and exploratory factor analysis. The results showed that 23 question items encompassed five components of accounting teacher professional identity; Cultural Knowledge (pedagogical cultural identity), Blending (accommodating students' purposes for school in the learning objectives), Identity Experiencing (by the experience of working life in the past, present, and individual expectations in the future in accounting work setting), Inter-Personal Skill, and Active in Professional Communities. The scale development requires continuous development tailing various new findings in the teacher professional identity and accountant professional identity.</p>

Performance of binary prediction models in high-correlation low-dimensional settings: a comparison of methods

Background Clinical prediction models are developed widely across medical disciplines. When predictors in such models are highly collinear, unexpected or spurious predictor-outcome associations may occur, thereby potentially reducing face-validity of the prediction model. Collinearity can be dealt with by exclusion of collinear predictors, but when there is no a priori motivation (besides collinearity) to include or exclude specific predictors, such an approach is arbitrary and possibly inappropriate. Methods We compare different methods to address collinearity, including shrinkage, dimensionality reduction, and constrained optimization. The effectiveness of these methods is illustrated via simulations. Results In the conducted simulations, no effect of collinearity was observed on predictive outcomes (AUC, R2, Intercept, Slope) across methods. However, a negative effect of collinearity on the stability of predictor selection was found, affecting all compared methods, but in particular methods that perform strong predictor selection (e.g., Lasso). Methods for which the included set of predictors remained most stable under increased collinearity were Ridge, PCLR, LAELR, and Dropout. Conclusions Based on the results, we would recommend refraining from data-driven predictor selection approaches in the presence of high collinearity, because of the increased instability of predictor selection, even in relatively high events-per-variable settings. The selection of certain predictors over others may disproportionally give the impression that included predictors have a stronger association with the outcome than excluded predictors.

Development and Validation of a Knowledge, Attitude and Practice Questionnaire on Antibiotic Use in Arabic and French Languages in Lebanon

Objectives: Validated knowledge–attitude–practice (KAP) questionnaires are essential to design and evaluate intervention programs on antibiotic use. Recently, we validated the first KAP questionnaire on antibiotics in Spain. Cross-cultural adaptation and validation of research tools increase their universal usefulness. Here, we aimed to validate the questionnaire in a developing country with different socioeconomic characteristics from that of Spain. Methods: We translated the previously developed KAP-questionnaire into Arabic and French, tailored it and then validated it in adult population in Lebanon. The item content validity index (I-CVI), scale content validity index (S-CVI/Ave) and modified Kappa (k*) were calculated. The construct validity of the questionnaire was evaluated using confirmatory factorial analysis (CFA, N = 1460) and its reliability was assessed using intraclass correlation coefficients (ICC, N = 100) and Cronbach’s alpha statistic. Results: ICV-I (>0.78), k* (equal to ICV-I for all items) and S-CVI/Ave (≥0.95) confirmed the questionnaire content validity. Pilot testing (N = 40) and face validity showed the understandability of the questionnaire by the population. Test–retest reliability analysis (N = 100) yielded ICC ≥ 0.59 for all knowledge and attitude items, showing the capacity of the questionnaire to generate reproducible results. CFA evidenced adequate fit of the chosen model, thus establishing the construct validity of the questionnaire (root mean squared error approximation = 0.053, standardized root mean square residual = 0.045, comparative fit index = 0.92 and Tucker–Lewis index = 0.90). The questionnaire showed an acceptable internal reliability (Cronbach’s alpha = 0.62) and was highly accepted in Lebanon (response rate = 96% and item response rates ≥ 94%). Conclusions: The validity of the KAP-questionnaire on antibiotics in Arabic and French was demonstrated in Lebanon.

Face Validity of Observed Meal Patterns Reported with 7-Day Diet Diaries in a Large Population-Based Cohort Using Diurnal Variation in Concentration Biomarkers of Dietary Intake

In a cross-sectional analysis of a population-based cohort (United Kingdom, N = 21,318, 1993–1998), we studied how associations between meal patterns and non-fasting triglyceride and glucose concentrations were influenced by the hour of day at which the blood sample was collected to ascertain face validity of reported meal patterns, as well as the influence of reporting bias (assessed using formula of energy expenditure) on this association. Meal size (i.e., reported energy content), mealtime and meal frequency were reported using pre-structured 7-day diet diaries. In ANCOVA, sex-specific means of biomarker concentrations were calculated by hour of blood sample collection for quartiles of reported energy intake at breakfast, lunch and dinner (meal size). Significant interactions were observed between breakfast size, sampling time and triglyceride concentrations and between lunch size, sampling time and triglyceride, as well as glucose concentrations. Those skipping breakfast had the lowest triglyceride concentrations in the morning and those skipping lunch had the lowest triglyceride and glucose concentrations in the afternoon, especially among acceptable energy reporters. Eating and drinking occasion frequency was weakly associated with glucose concentrations in women and positively associated with triglyceride concentrations in both sexes; stronger associations were observed for larger vs. smaller meals and among acceptable energy reporters. Associations between meal patterns and concentration biomarkers can be observed when accounting for diurnal variation and underreporting. These findings support the use of 7-day diet diaries for studying associations between meal patterns and health.

Indonesian Dental Students' Attitudes, Knowledge, Preparation, and Willingness to Treat HIV/AIDS Patients

Objective To assess the Indonesian dental students' knowledge of HIV/AIDS in terms of transmission and oral manifestation, the attitudes toward people living with HIV/AIDS (PLWHA), the preparedness in infection control, and willingness to treat PLWHA, and assess the factors for willingness to treat PLWHA. Materials and Methods A modified version of a questionnaire used to assess dental students' knowledge, attitude, preparedness, and willingness to treat PLWHA in China was used. The questionnaire was cross-culturally adapted into Indonesian and had been pre-tested for face validity and test and retest reliability. The dental students from 32 dental schools in Indonesia were invited to participate in the study. Results A total of 1,280 dental students from 23 dental schools participated in the study. This study found that only 63% of students scored higher than 70% for knowledge of HIV/AIDS, and the mean score for knowledge was 15.02 (2.4). Higher than 80% of students had a positive professional attitude toward PWLHA; however, 80% of students worried about possible disease transmission in the dental office by PLWHA and 70% of students overestimated the occupational risk when treating PLWHA. The dental students had good preparedness for infection control with a mean score for preparedness of 3.19 (0.4). The mean score for willingness was 2.5 (0.9). Willingness to treat significantly differed by the type of universities, gender, age, and clinical experience. This study showed that knowledge about HIV/AIDS correlated with the willingness to treat PLWHA among dental students. Conclusion Dental students who have good knowledge about HIV infection tend to have a good willingness to treat PLWHA. The knowledge would in turn affect their attitude, preparedness, and willingness to provide care for PLWHA with confidence and comfort. This study suggests that the improvement of dental training may play an important role in changing students' perception of willingness to treat PLWHA.

Adaptation and Preliminary Validation of the Arabic Version of the Role Checklist Version 3 (RCv3)

Arab occupational therapists are in need either to develop their own outcome measures or to translate well-known outcome measures to their culture. The objective of this study was to translate, cross-culturally adapt, and validate preliminarily the Role Checklist Version 3 (RCv3) into Arabic language. The translation and cross-cultural adaptation process of the RCv3 into Arabic language was conducted in four main steps: (a) forward translation, (b) back translation, (c) expert panel review process, and (d) psychometric testing (Validity). The expert panel established the face validity and the content validity of the Arabic RCv3. Pilot testing was conducted, and the utility and content validity of the tool were confirmed. The final version of the Arabic RCv3 is now available for use in the Arab countries. However, further psychometric properties including reliability and responsiveness of this tool is warranted, thus supporting its wide use and proper application.

The Translation and Validation of The Smartphone Use Questionnaire (SUQ) Into The Malay Language

Interruptions caused by frequent smartphone use steals attention away from daily activities, bringing serious implications onto an individual’s health, safety and education. Smartphone Use Questionnaire (SUQ) is a 20-item questionnaire developed to assess the pattern of smartphone use and its effect on attention. This study was done to translate and validate the Malay-language version of the SUQ and to measure the psychometric properties of the Malay-version SUQ to justify its use in Malaysia. A forward and back-translation was done by four individuals, who were three physicians and one linguist. Content and face validity was done involving three experts who were a linguist, psychiatrist and epidemiologist. Psychometric testing was conducted on a sample of 195 individuals proficient in the Malay language. A construct validity test was performed using factor analysis and the internal reliability was tested by calculating for the Cronbach’s Alpha. The age range of the sample was 13-59 years, most of which were female and of the Malay race. Using principal component analysis with direct oblimin rotation, the factor analysis extracted two components similar to the original study: General Use and Absent-Minded Use. However, question number 20 was grouped into General Use component, whereas in the original study it was under the Absent- Minded Use component. The Cronbach’s Alpha for the obtained components was 0.884 and 0.927, respectively. This study found that the Malay-version SUQ was a valid and reliable instrument for use in Malaysia in assessing inattention associated with smartphone use.

Validity and Reliability of the Persian Versions of Primary and Secondary Screening Instrument for Targeting Educational Risk Questionnaires

Background and Aim: Hearing loss in children leads to speech and language delays, low academic achievement, literacy delays, and psychosocial difficulties. Screening instrument for targeting educational risk (SIFTER) is one of the questionnaires used for evaluation of students’ performance in schools. The current study aims to develop Persian versions of primary and secondary SIFTER questionnaires and assessing their validity and reliability. Methods: The main English versions of primary and secondary SIFTER questionnaires were translated into Persian named as P-SIFTER and secondary P-SIFTER. Then, their face validities were determined based on the options of related experts. The final versions were completed by 55 teachers of 150 students (64 primary and 86 secondary school students) divided into two groups of hearing-impaired (HI) and normal-hearing (NH) students. The test- retest reliabilities were assessed in 117 students (64 primary and 53 secondary school students). Results: The results revealed that these questionnaires had high face validity. The content validity index for P-SIFTER and secondary P-SIFTER were obtained 0.94 and 0.92, respectively. The total score of P-SIFTER was 51.85 and 65.41 in HI and NH students, respectively. For the secondary P-SIFTER, it was 58.75 and 67.48, respectively. The test-retest reliability showed high correlation for NH and HI students between P-SIFTER and secondary P-SIFTER scores. The Cronbach’s alpha value for the overall score of P-SIFTER was 0.96 for both HI and NH students; for secondary P-SIFTER, the values were 0.94 and 0.93, respectively. Conclusion: The Persian versions of primary and secondary SIFTER questionnaires have acceptable validity and reliability.