Tools for Evaluating the Implementation of Complex Education Interventions

2021 ◽  
pp. 109821402095849
Author(s):  
Kirsten Kainz ◽  
Allison Metz ◽  
Noreen Yazejian
Keyword(s):  
Learning Outcomes ◽  
Complex Intervention ◽  
Large Scale ◽  
Complex Interventions ◽  
Complexity Science ◽  
Space Complexity ◽  
Systems Science ◽  
Multiple Components ◽  
Education Interventions

Large-scale education interventions aimed at diminishing disparities and generating equitable learning outcomes are often complex, involving multiple components and intended impacts. Evaluating implementation of complex interventions is challenging because of the interactive and emergent nature of intervention components. Methods that build from systems science have proven useful for addressing evaluation challenges in the complex intervention space. Complexity science shares some terminology with systems science, but the primary aims and methods of complexity science are different from those of systems science. In this paper we describe some of the language and ideas used in complexity science. We offer a set of priorities for evaluation of complex interventions based on language and ideas used in complexity science and methodologies aligned with the priorities.

Evaluating Large-Scale STEM Outreach Efficacy with a Consistent Theme: Thermodynamics for Elementary School Students

2018 ◽  
Author(s):  
Joseph DeWilde ◽  
Esha Rangnekar ◽  
Jeffrey Ting ◽  
Joseph Franek ◽  
Frank S. Bates ◽  
...  
Keyword(s):  
Elementary School ◽  
Learning Outcomes ◽  
Elementary School Students ◽  
Large Scale ◽  
College Education ◽  
School Students ◽  
Central Concept ◽  
Specific Goal ◽  
Survey System ◽  
Society Today

A biannual chemistry demonstration-based show named “Energy and U” was created to extend the general outreach themes of STEM fields and a college education with a specific goal: to teach the First Law of Thermodynamics to elementary school students. Energy is a central concept in chemical education, most STEM disciplines, and it is the concept at the foundation of many of the greatest challenges faced by society today. The effectiveness of the program was analyzed using a clicker survey system. This study provides one of the first examples of incorporating real-time feedback into large- scale chemistry-based outreach events for elementary school students in order to quantify and better understand the broader impact and learning outcomes.

PENGEMBANGAN MEDIA BIG BOOK UNTUK MENINGKATKAN HASIL BELAJAR PECAHAN SENILAI SISWA SD

2018 ◽  
Vol 16 (1) ◽  
pp. 67-76
Author(s):  
Disyacitta Neolia Firdana ◽  
Trimurtini Trimurtini
Keyword(s):  
Learning Outcomes ◽  
Large Scale ◽  
Fourth Grade ◽  
Learning Activities ◽  
Equivalent Fractions ◽  
Fourth Grade Students ◽  
Teacher Needs ◽  
The Media ◽  
Test Result ◽  
Better Than

This research aimed to determine the properness and effectiveness of the big book media on learning equivalent fractions of fourth grade students. The method of research is Research and Development  (R&D). This study was conducted in fourth grade of SDN Karanganyar 02 Kota Semarang. Data sources from media validation, material validation, learning outcomes, and teacher and students responses on developed media. Pre-experimental research design with one group pretest-posttest design. Big book developed consist of equivalent fractions material, students learning activities sheets with rectangle and circle shape pictures, and questions about equivalent fractions. Big book was developed based on students and teacher needs. This big book fulfill the media validity of 3,75 with very good criteria and scored 3 by material experts with good criteria. In large-scale trial, the result of students posttest have learning outcomes completness 82,14%. The result of N-gain calculation with result 0,55 indicates the criterion “medium”. The t-test result 9,6320 > 2,0484 which means the average of posttest outcomes is better than the average of pretest outcomes. Based on that data, this study has produced big book media which proper and effective as a media of learning equivalent fractions of fourth grade elementary school.

Statistical methods for unidirectional switch designs: Past, present, and future

2017 ◽  
Vol 27 (9) ◽  
pp. 2872-2882 ◽  
Author(s):  
Zhuozhao Zhan ◽  
Geertruida H de Bock ◽  
Edwin R van den Heuvel
Keyword(s):  
Statistical Methods ◽  
Large Scale ◽  
Sample Size Calculation ◽  
Complex Interventions ◽  
New Drugs ◽  
Control Treatment ◽  
Future Research ◽  
Control Group ◽  
Unified Framework ◽  
New Treatment

Clinical trials may apply or use a sequential introduction of a new treatment to determine its efficacy or effectiveness with respect to a control treatment. The reasons for choosing a particular switch design have different origins. For instance, they may be implemented for ethical or logistic reasons or for studying disease-modifying effects. Large-scale pragmatic trials with complex interventions often use stepped wedge designs (SWDs), where all participants start at the control group, and during the trial, the control treatment is switched to the new intervention at different moments. They typically use cross-sectional data and cluster randomization. On the other hand, new drugs for inhibition of cognitive decline in Alzheimer’s or Parkinson’s disease typically use delayed start designs (DSDs). Here, participants start in a parallel group design and at a certain moment in the trial, (part of) the control group switches to the new treatment. The studies are longitudinal in nature, and individuals are being randomized. Statistical methods for these unidirectional switch designs (USD) are quite complex and incomparable, and they have been developed by various authors under different terminologies, model specifications, and assumptions. This imposes unnecessary barriers for researchers to compare results or choose the most appropriate method for their own needs. This paper provides an overview of past and current statistical developments for the USDs (SWD and DSD). All designs are formulated in a unified framework of treatment patterns to make comparisons between switch designs easier. The focus is primarily on statistical models, methods of estimation, sample size calculation, and optimal designs for estimation of the treatment effect. Other relevant open issues are being discussed as well to provide suggestions for future research in USDs.

scholarly journals Long-Term Effect of Interactive Online Dietician Weight Loss Advice in General Practice (LIVA) Protocol for a Randomized Controlled Trial

2014 ◽  
Vol 2014 ◽  
pp. 1-6 ◽  
Author(s):  
Carl J. Brandt ◽  
Vibeke Brandt ◽  
Mathilde Pedersen ◽  
Dorte Glintborg ◽  
Søren Toubro ◽  
...  
Keyword(s):  
Weight Loss ◽  
General Practice ◽  
Large Scale ◽  
Controlled Trial ◽  
Complex Interventions ◽  
Control Group ◽  
Average Weight ◽  
Practice Setting ◽  
Management Support ◽  
General Practice Setting

Background. Internet-based complex interventions aiming to promote weight loss and optimize healthy behaviors have attracted much attention. However, evidence for effect is lacking. Obesity is a growing problem, resulting in an increasing demand for cost efficient weight loss programs suitable for use on a large scale, for example, as part of standard primary care. In a previous pilot project by Brandt et al. (2011) without a control group, we examined the effects of online dietician counseling and found an average weight loss of 7.0 kg (95% CI: 4.6 to 9.3 kg) after 20 months. Aims and Methods. To analyze the effects of a complex intervention using trained dieticians in a general practice setting combined with internet-based interactive and personalized weight management support compared with conventional advice with a noninteractive internet support as placebo treatment in 340 overweight patients during a 2-year period. Primary endpoints are weight loss and lowering of cholesterol (LDL). We will also explore patients’ sociodemographics and use of the intervention as well as the health professionals’ views and perceptions of the intervention (their role and the advice and support that they provide). Perspective. The project will generate knowledge on the cost-effectiveness of a complex internet-based intervention in a general practice setting and on barriers and acceptability among professionals and patients.

scholarly journals Development of Biomagz Based on Local Wisdom on Biodiversity Material in High School

2019 ◽  
Vol 8 (2) ◽  
pp. 246-256
Author(s):  
Rina Wahyuningsih ◽  
Sri Sukaesih ◽  
Endah Peniati
Keyword(s):  
Experimental Design ◽  
Learning Outcomes ◽  
Large Scale ◽  
Student Assessment ◽  
Average Score ◽  
Small Scale ◽  
Learning Resources ◽  
Environmental Care ◽  
High Category ◽  
Post Test

This study aims to describe the types of learning resources used previously at SMA N 3 Salatiga, describe the feasibility of Biomagz Based on Local Wisdom, and test the effectiveness of Biomagz Based on Local Wisdom on learning outcomes and student’s environmental care attitude. The method used is research development (R & D). Products are validated by material and media validators, and revised before being tested. Small-scale trials (readability test) use 10 students from class X MIPA 1, while large-scale trials use class X MIPA 2 with the experimental design Pre-experimental Design with the type of Pre-test and Post-test One Group Design. The results of the study show the variety of learning resources used previously including biological, teacher, internet, biology books, and the environment around students. The Feasibility of Biomagz Based on Local Wisdom obtained an average score of 92.21% with very decent criteria. Large-scale trials with an average N-gain value 0.49 in the medium category, the average classical completeness is 85.29%, and the environment care attitude of students is based on the observation result of 92.9% with a high category and the result of the inter-student assessment is 86, 18% with a high category. Based on the results of this study, it can be concluded that Biomagz Based Local Wisdom is well worth and effective on learning outcomes and student’s environmental care attitude.

scholarly journals PENGARUH PENGGUNAAN VIDEO PEMBELAJARAN DAN MINAT BELAJAR TERHADAP HASIL BELAJAR MATA KULIAH PENGANTAR MANAJEMEN PADA PROSES PEMBELAJARAN JARAK JAUH DI MASA PANDEMI COVID 19.

2021 ◽  
Vol 5 (2) ◽  
Author(s):  
Siti Sahara ◽  
Sugeng Priyanto
Keyword(s):  
Distance Learning ◽  
Learning Outcomes ◽  
Learning Process ◽  
Large Scale ◽  
Classical System ◽  
Learning System ◽  
Student Interest ◽  
Study Program ◽  
Ex Post ◽  
Data Collection Technique

Currently, Covid-19 is a hot topic of conversation in all countries, including Indonesia. In Indonesia itself, a Large-Scale Social Restriction (PSBB) policy was implemented to suppress the spread of this virus so that all activities carried out outside the home must be stopped until this pandemic has subsided. The imposition of Large-Scale Social Restrictions (PSBB) made the learning system change from a classical system to a Distance Learning (PJJ) or online system. In the Introduction to Management course, student participation was found to be low during the learning process. Lecturers have difficulty determining whether students take learning seriously or not. Often in distance learning, some students deliberately put up a video that has been recorded, so that it seems like they are following the learning process, but in reality, they are doing something else. This is caused by several factors, including media which can be in the form of video lessons and student interest in learning. Looking at the description above, researchers are interested in conducting research that aims to determine the effect of using video learning and learning interest on learning outcomes in the Management Introduction course in the distance learning process during the Covid 19 pandemic. This research was conducted in the D3 Transportation Study Program, Faculty of Engineering, the State University of Jakarta in semester 113 of the 2020/2021 school year. This research is an ex post facto research with a quantitative approach. The number of samples in this study consisted of 76 students. The data collection technique is done by using questionnaires and documentation. The data obtained were analyzed using a multiple regression test. The results of this study indicate that Fcount ≥ Ftable = 13.53 ≥ 3.124 so it can be concluded that there is a significant influence between the use of instructional videos (X1) and interest (X2) on learning outcomes in the Introduction to Management (Y) course. For this reason, the role of lecturers is indispensable in continuously making creative innovations in developing various kinds of learning media that can increase student interest in learning during the Covid 19 pandemic. 

Methodological Challenges and Solution Strategies During Implementation of a Midwife-led Multicentre Randomized-controlled Trial (RCT) in Maternity Hospitals

2021 ◽  
Author(s):  
Sabine Striebich ◽  
Elke Mattern ◽  
Theresa Oganowski ◽  
Rainhild Schäfers ◽  
Gertrud M. Ayerle
Keyword(s):  
Complex Intervention ◽  
Controlled Trial ◽  
Complex Interventions ◽  
Quality Standard ◽  
Quality Data ◽  
Technical Failure ◽  
Maternity Hospitals ◽  
Study Teams ◽  
Randomised Controlled ◽  
Delivery Rooms

Abstract Background: RCTs with complex interventions are methodically challenging. Careful planning under everyday conditions in compliance with the relevant international quality standard (ICH-GCP guideline) is crucial. Specific challenges exist for RCTs conducted in delivery rooms due to various factors that cannot be planned beforehand, such as “peak hours” of births and a high work burden for midwives and obstetricians. Moreover, in Germany as well as in other countries, midwives and obstetricians have frequently little experience as investigators in clinical trials. Methods: The randomised controlled trial “BE-UP” tests the effectiveness of an alternative birthing room on the rate of vaginal births and woman-oriented outcomes. In the process of implementing the trial in 17 obstetrical units and in the endeavour to reach the calculated sample size of 3,800 women, the research team encountered a variety of unexpected challenges. The aim is to describe in greater detail the methodical and organisational challenges and to inform about the research team’s strategies to overcome them.Results:The results are presented in five sectors: 1) Selection of and support for cooperating hospitals: they are to be selected according to predefined criteria and strategies to offer continuous support in trial implementation must be mapped out.2) Establishing a process of requesting informed consent: a quality-assured process to inform pregnant women early on must be feasible and effective. 3) Individual, digital real time randomization: besides instructing the maternity teams appropriate measures for technical failure must be provided.4) The standardized birthing room: the complex intervention is to be implemented according to study protocol, yet adapted to the prevailing conditions in the delivery rooms.5) GCP-compliant documentation: midwives and obstetricians is to be instructed in high quality data collection, supported by external monitoring throughout the trial.Conclusion:Since not all potential challenges can be anticipated in the planning of a trial, study teams need to be flexible and react promptly to any problems that threaten recruitment or the implementation of the complex intervention. Thought should be given to the perspectives of midwives and obstetricians as recruiters and how clinic-intern processes could be adapted to correspond with the trial’s requirements.

Introduction: A ‘fourth age’ of psychiatric epidemiology?

2020 ◽  
pp. 1-4
Author(s):  
Jayati Das-Munshi ◽  
Tamsin Ford ◽  
Matthew Hotopf ◽  
Martin Prince ◽  
Robert Stewart
Keyword(s):  
Healthcare Service ◽  
Large Scale ◽  
Complex Interventions ◽  
Epidemiological Studies ◽  
Wearable Devices ◽  
Psychiatric Epidemiology ◽  
High Quality ◽  
Oxford University ◽  
Large Scale Data ◽  
Scale Data

This is the introduction to the second edition of ‘Practical Psychiatric Epidemiology’ published by Oxford University Press. In this introduction the Editors reflect on developments since the first edition. Themes touched upon include the ongoing need for high quality descriptive data, the contribution of wearable devices and technologies to generating data for psychiatric epidemiological studies, developments relating to the availability of large-scale data resources or so-called ‘big data’ in psychiatric epidemiology, ongoing issues relating to accurate measurement in psychiatric epidemiology and the contribution of complex interventions to effective healthcare service delivery, and in particular, the way in which these are effectively implemented. The chapter concludes with a reflection on the continued importance of psychiatric epidemiology to the field of mental health.

scholarly journals A worked example of initial theory-building: PARTNERS2 collaborative care for people who have experienced psychosis in England

Evaluation ◽  
2019 ◽  
Vol 26 (1) ◽  
pp. 6-26
Author(s):  
Ruth Gwernan-Jones ◽  
Nicky Britten ◽  
Jon Allard ◽  
Elina Baker ◽  
Laura Gill ◽  
...  
Keyword(s):  
Focus Groups ◽  
Complex Intervention ◽  
Collaborative Care ◽  
Iterative Process ◽  
Complex Interventions ◽  
Theory Building ◽  
Service Users ◽  
Programme Theory ◽  
Worked Example ◽  
Care Intervention

In this article, we present an exemplar of the initial theory-building phase of theory-driven evaluation for the PARTNERS2 project, a collaborative care intervention for people with experience of psychosis in England. Initial theory-building involved analysis of the literature, interviews with key leaders and focus groups with service users. The initial programme theory was developed from these sources in an iterative process between researchers and stakeholders (service users, practitioners, commissioners) involving four activities: articulation of 442 explanatory statements systematically developed using realist methods; debate and consensus; communication; and interrogation. We refute two criticisms of theory-driven evaluation of complex interventions. We demonstrate how the process of initial theory-building made a meaningful contribution to our complex intervention in five ways. Although time-consuming, it allowed us to develop an internally coherent and well-documented intervention. This study and the lessons learnt provide a detailed resource for other researchers wishing to build theory for theory-driven evaluation.

Research on the Application of Organizational Management Information System

2014 ◽  
Vol 687-691 ◽  
pp. 2853-2857
Author(s):  
Zhi Mei Zhang
Keyword(s):  
Information System ◽  
Management Information System ◽  
Large Scale ◽  
Organizational Management ◽  
Systems Science ◽  
Management Information ◽  
The Core ◽  
Core Function ◽  
Science Management ◽  
Sport Events

To well organize China's Large-scale Sport Events, causes the Large-scale Sport Events organization to manage in the process each kind of activity and the work carries on the effective classification the equipment personnel and the division of labor authorization, we synthesized research techniques and so on utilization systems science, management science, mathematics modeling, artificial intelligence, information technology design in China's Large-scale Sport Events to organize the management information system, This article comprehensively elaborated this system function demand, the bare bone, the system design and the core function and so on.

Sign in / Sign up

Export Citation Format

Share Document