Defining and measuring completion and assessment biases with respect to English language and development status: not all MOOCs are equal

The emergence of Massive Open Online Courses (MOOCs) broadened the educational landscape by providing free access to quality learning materials for anyone with a device connected to the Internet. However, open access does not guarantee equals opportunities to learn, and research has repetitively reported that learners from affluent countries benefit the most from MOOCs. In this work, we delve into this gap by defining and measuring completion and assessment biases with respect to learners’ language and development status. We do so by performing a large-scale analysis across 158 MITx MOOC runs from 120 different courses offered on edX between 2013 and 2018, with 2.8 million enrollments. We see that learners from developing countries are less likely to complete MOOCs successfully, but we do not find evidence regarding a negative effect of not being English-native. Our findings point out that not only the specific population of learners is responsible for this bias, but also that the course itself has a similar impact. Independent of and less frequent than completion bias, we found assessment bias, that is when the mean ability gained by learners from developing countries is lower than that of learners from developed countries. The ability is inferred from the responses of the learners to the course-assessment using item response theory (IRT). Finally, we applied differential item functioning (DIF) methods with the objective of detecting items that might be causing the assessment bias, obtaining weak, yet positive results with respect to the magnitude of the bias reduction. Our results provide statistical evidence on the role that course design might have on these biases, with a call for action so that the future generation of MOOCs focus on strengthening their inclusive design approaches.

Defining and Measuring Completion and Assessment Biases with Respect to English Language and Development Status: Not All MOOCs are Equal

The emergence of Massive Open Online Courses (MOOCs) broadened the educational landscape by providing free access to quality learning materials for anyone with a device connected to the Internet. However, open access does not guarantee equals opportunities to learn, and research has repetitively reported that learners from affluent countries benefit the most from MOOCs. In this work, we delve into this gap by defining and measuring completion and assessment biases with respect to learners' language and development status. We do so by performing a large-scale analysis across 158 MITx MOOC runs from 120 different courses offered on edX between 2013 and 2018, with 2.8 million enrollments. We see that learners from developing countries are less likely to complete MOOCs successfully, but we do not find evidence regarding a negative effect of not being English-native. Our findings point out that not only the specific population of learners is responsible for this bias, but also that the course itself has a similar impact. Independent of and less frequent than completion bias, we found assessment bias, that is when the mean ability gained by learners from developing countries is lower than that of learners from developed countries. The ability is inferred from the responses of the learners to the course-assessment using Item Response Theory (IRT).Finally, we applied Differential Item Functioning (DIF) methods with the objective of detecting items that might be causing the assessment bias, obtaining weak, yet positive results with respect to the magnitude of the bias reduction. Our results provide statistical evidence on the role that course design might have on these biases, with a call for action so that the future generation of MOOCs focus on strengthening their inclusive design approaches.

Adherence to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist in articles published in EAACI Journals: a bibliographic study.

Research data derived from observational studies are accumulating quickly in the field of allergy and immunology and a large amount of observational studies are published every year. The aim of the present study was to evaluate the adherence to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist by papers published in the three European Academy of Allergy and Clinical Immunology journals, during the period 2009-2018. To this end, we conducted a bibliographic study of up to eight randomly selected papers per year per Journal. Our literature search resulted in 223 papers. Among those, 80, 80 and 63 records were from Pediatric Allergy and Immunology, Allergy and Clinical and Translational Allergy, respectively; the latter was published only from 2011 on. Prospective, case-control, and cross-sectional designs were described in 88, 43, and 92 papers, respectively. Full reporting of all STROBE items was present in 47.4%, 45.6%, and 41.2% for the cohort, cross-sectional, and case-control studies, respectively. Generally, no time trend in adherence of reporting STROBE items was observed, apart from reporting funding, which increased from 60% in 2009/2010 to more than 90% in 2018. We identified a cluster of STROBE items with low proportions of full reporting constituted by the items on reporting study design in the title and methods, variables types along with their measurement/assessment, bias and confounding, study size, and grouping of variables. It appears that the STROBE checklist is a suitable tool in observational allergy epidemiology. However, adherence to the STROBE checklist appeared suboptimal.

How to Interpret an Investigator’s Brochure for Meaningful Risk Assessment: Results of an AGAH Discussion Forum

Purpose A discussion forum was hosted by the Association for Applied Human Pharmacology (AGAH e.V.) to critically debate how to interpret and optimise the Investigator’s Brochure (IB) for meaningful risk assessment of early clinical trials. Materials and Methods Four topics were specifically discussed: deficiencies/uncertainties in IBs, guidance for the investigator, reference safety information, and potential risks for human subjects associated with inadequate non-clinical safety assessment in the IB. In each case, 43 participants took part in a real-time online survey with pre-defined questions to capture the audience’s opinion. Results The ‘Summary of Data and Guidance for the Investigator’ was considered as the section of the IB with the highest need for improvement with emphasis on readability, comprehensibility, timeliness of data, and appropriateness for risk assessment. It was suggested that the IB should at least be signed by the sponsor’s scientist responsible for the content on pharmacology and toxicology. It was agreed that sponsors should consider thoroughly whether changes to an IB constitute a substantial amendment, and that the IB should include a section on the change history. Non-clinical pharmacology studies with negative outcomes should be reported in the IB in order to avoid assessment bias. The reference safety information for expectedness assessment of suspected serious adverse reactions should be provided as a stand-alone section of the IB. Conclusion The overall consensus was that an optimised presentation of data will ensure the best possible understanding of a compound’s characteristics and an optimal benefit-risk assessment which will safeguard the participants in clinical trials.

Multi-group confirmatory factor analysis of the life attitude profile in Chilean adolescents

Adolescents are challenged to take decisions about issues associated to the vital circle and the comprehension of the world. Meaning in life (MiL) takes a leading role in adolescence. Reker and Peacock developed the Life Attitude Profile (LAP) for assessing MiL, and Erci developed a revised version (LAP-R). The main objective of this study was to analyze the factorial structure, reliability, invariance across gender, and absence/presence of assessment bias of a Spanish adaptation of the Turkish LAP-R among 2138 Chilean adolescents (1205 boys, 912 girls, and 8 others), who were studying in 3rd and 4th year. The LAP-R showed an acceptable structural validity, acceptable internal consistency, and invariance across gender. MiL as a significant domain of human experience in adolescents.

Deliberate Practice Makes Perfect: Improving Therapist Effectiveness

Psychotherapy works. The results of numerous studies show that those individuals treated are better off than those not treated or on waitlists with an average effect size of .8 (Duncan et al., 2008). To put it in perspective, the effects of psychotherapy are equal to those found for coronary artery bypass surgery and 4 times greater than fluoride in the prevention dental cavities. Yet, three persistent problems plague the psychotherapy field: 1) clients drop out of therapy at alarming rates – almost half of clients decide not to continue and prematurely terminate; 2) not only do therapists not notice when clients are at risk for dropping out, they also do not detect when things are getting worse (approximately 10% of clients get worse after starting therapy); and 3) a small percentage of clients (10%) accounts for the largest amount of expenditures (Minami, 2008). This last finding may be the result of therapists not realizing when things are not working or getting worse and instead of changing course, doing more of what is not working, over and over again. Along these lines, most therapists do not have an accurate sense of their helpfulness and on average, overrate their effectiveness by 65% (Chow, 2014). Given the issues with retention, coupled with the self-assessment bias among therapists, it’s not surprising that psychotherapy outcomes have not appreciably improved over the past 40 years.

Re-evaluating Atlantic cod mortality including lobster bycatch: where could we be today?

Full accounting of fisheries mortality is one of the most tractable ways to improve stock assessments. However, it can be challenging to obtain in cases when missing catch comes from small-scale nontarget fisheries unrequired to report incidental catch. Atlantic cod (Gadus morhua) in the Gulf of Maine (GoM), USA, once served as a regionally important fishery, but has been serially depleted to <5% of historic spawning stock biomass. Recent management efforts to rebuild GoM cod have largely failed. We test the hypothesis that unaccounted bycatch of Atlantic cod in the Maine American lobster (Homarus americanus) fishery is a substantial missing piece in the GoM Atlantic cod assessment. We integrated multiple scenarios of hind-casted discards into the two accepted regional cod assessment models from 1982 to 2016. Incorporation of discards improved the assessment bias for both models (10%–15%), increased estimates of spawning stock biomass (4%), and decreased estimates of fishing mortality (9%). A novel evaluation of longitudinal model bias suggests that alternative modelling approaches or specifications may be warranted. We highlight the importance of accounting for all fishery-related mortality and the need for methods to deliver more comprehensive estimates from both target and nontarget fisheries.