Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency

Purpose Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to “Pass/Fail” removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS). Design Retrospective analysis of application characteristics versus residency performance was conducted. Participants Residents who graduated from the Dean McGee Eye Institute/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study. Methods Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS. Main Outcome Measures Characteristics from residency applications that correlate with resident performance were the primary outcome of this study. Results Fifty-five residents and 21 faculty members were included. A grade of “A” or “Honors” in the obstetrics/gynecology (OB/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance. Conclusions Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking.

Using mobile technology in assessment of entrustable professional activities in undergraduate medical education

Background The adoption of competency-based medical education requires objective assessments of a learner’s capability to carry out clinical tasks within workplace-based learning settings. This study involved an evaluation of the use of mobile technology to record entrustable professional activity assessments in an undergraduate clerkship curriculum. Approach A paper-based form was adapted to a mobile platform called eClinic Card. Students documented workplace-based assessments throughout core clerkship and preceptors confirmed accuracy via mobile phones. Assessment scores for the 2017–2018 academic year were collated and analyzed for all core rotations, and preceptors and students were surveyed regarding the mobile assessment experience. Evaluation The mobile system enabled 80 students and 624 preceptors to document 6850 assessment submissions across 47 clinical sites over a 48-week core clerkship curriculum. Students’ scores demonstrated progressive improvement across all entrustable professional activities with stage-appropriate levels of independence reported by end of core clerkship. Preceptors and students were satisfied with ease of use and dependability of the mobile assessment platform; however, students felt quality of formative coaching feedback could be improved. Reflection Our preliminary evaluation suggests the use of mobile technology to assess entrustable professional activity achievement across a core clerkship curriculum is a feasible and acceptable modality for workplace-based assessment. The use of mobile technology supported a programmatic assessment approach. However, meaningful coaching feedback, as well as faculty development and support, emerged as key factors influencing successful adoption and usage of entrustable professional activities within an undergraduate medical curriculum.

Effect of a high value care curriculum on standardized patient exam in the Core Clerkship in Internal Medicine

Background With almost 20% unnecessary spending on healthcare, there has been increasing interest in high value care defined as the best care for the patient, with the optimal result for the circumstances, delivered at the right price. The American Association of Medical Colleges recommend that medical students are proficient in concepts of cost-effective clinical practice by graduation, thus leading to curricula on high value care. However little is published on the effectiveness of these curricula on medical students’ ability to practice high value care. Methods In addition to the standard curriculum, the intervention group received two classroom sessions and three virtual patients focused on the concepts of high value care. The primary outcome was number of tests and charges for tests on standardized patients. Results 136 students enrolled in the Core Clerkship in Internal Medicine and 70 completed the high value care curriculum. There were no significant differences in ordering of appropriate tests (3.1 vs. 3.2 tests/students, p = 0.55) and inappropriate tests (1.8 vs. 2.2, p = 0.13) between the intervention and control. Students in the intervention group had significantly lower median Medicare charges ($287.59 vs. $500.86, p = 0.04) and felt their education in high value care was appropriate (81% vs. 56%, p = 0.02). Conclusions This is the first study to describe the impact of a high value care curriculum on medical students’ ordering practices. While number of inappropriate tests was not significantly different, students in the intervention group refrained from ordering expensive tests.

The Association With Physical Fitness and Academic Performance at America’s Military Medical School

Introduction Undergraduate and graduate medical education both have been reported to have high rates of depression, anxiety, and burnout as a result of the rigors of their educational curricula. Wellness, including physical fitness, is important to the profession because it may help mitigate these increased rates of depression and anxiety. While several studies examine physical fitness and academic performance in primary and secondary education, few studies have examined the association between fitness and performance in health professions education. Given the demands of medical school, this investigation aimed to investigate the possibility of an association between physical fitness and body mass index (BMI) with academic performance. Another goal was to examine the change in physical fitness and BMI throughout medical school. Materials and Methods Standardized measures of physical fitness were examined in 192 Army students from the Uniformed Services University of Health Sciences over 4 years from the classes of 2017–2019. Measurements include (1) body composition (BMI); (2) muscular endurance (timed push-ups); (3) aerobic endurance (timed run), and (4) total fitness defined as total points from sit-ups, run, and push-ups (everything but BMI). Pearson correlation analysis was conducted between physical fitness and academic performance measures such as United States Medical Licensing Examination (USMLE) Step 1, USMLE Step 2 Clinical Knowledge (CK), and average core clerkship National Board of Medical Examiners (NBME) exam scores. A repeated measures analysis of variance was conducted to examine the differences of students’ fitness total points and BMI over 4 fitness measurements spanning approximately 1.7 years. Results There were weak positive correlations, 0.20 and 0.22, between USMLE Step 2 CK exam scores and push-up points and total fitness points, respectively. There also were weak positive correlations, 0.21 and 0.24, between core clerkship average NBME exam scores and push-up points and total fitness points, respectively. No statistically significant change of individual students’ fitness total point change was observed (F(2.52, 334.93 = 1.37, P = 0.26)). There was also no individual BMI change (F(2.36, 274.21 = 2.78, P = .06)) over the first four assessment measurements (about 1.7 years lapse). However, the class means over the four measurements showed a decrease of fitness total points and an increase of BMI. Conclusions Our investigation suggests that there may be correlations between physical fitness and clinical exam performance such as USMLE Step 2 CK and average core clerkship NBME exams. Although there is no statistically significant change in individual fitness total points or BMI, the study suggests that physical fitness and BMI may decline during medical school. This may be as a result of increasing academic demands while balancing clinical duties from clerkship rotations.