Finland, A Package Deal: Disciplinary Climate in Science Classes, Science Dispositions and Science Literacy

Finland’s educational prowess, though tempered by recent international assessments, has remained intact. This report focused on lessons that could be learned regarding secondary-level science education from the Program for International Student Assessment (PISA) 2015, science-focused assessment. That PISA iteration included not only science literacy but also students’ science dispositions (epistemology, enjoyment, interest, and self-efficacy) and the schools’ science climate measures (disciplinary climate and teaching support). Due to the hierarchical nature of the PISA data, multilevel models were employed in this Finnish study, involving 5582 students from 167 schools. Science dispositions (as outcome measures) were differently associated with teaching support and disciplinary climate (epistemology with neither; enjoyment and interest, with both). Science literacy (as an outcome measure) was associated with all four science dispositions, whether modeled with each science disposition separately or all four simultaneously. Science literacy was also associated with the disciplinary climate in science classes for all tested models. We concluded that, in the Finnish context, science dispositions and the disciplinary climate were predictive of science literacy. Furthermore, we presented evidence from the literature indicating that these conclusions may well extend to other international contexts.

RALSA: the R analyzer for large-scale assessments

This paper presents the R Analyzer for Large-Scale Assessments (), a newly developed package for analyzing data from studies using complex sampling and assessment designs. Such studies are, for example, the IEA’s Trends in International Mathematics and Science Study and the OECD’s Programme for International Student Assessment. The package covers all cycles from a broad range of studies. The paper presents the architecture of the package, the overall workflow and illustrates some basic analyses using it. The package is open-source and free of charge. Other software packages for analyzing large-scale assessment data exist, some of them are proprietary, others are open-source. However, is the first comprehensive package, designed for the user experience and has some distinctive features. One innovation is that the package can convert SPSS data from large scale assessments into native data sets. It can also do so for PISA data from cycles prior to 2015, where the data is provided in tab-delimited text files along with SPSS control syntax files. Another feature is the availability of a graphical user interface, which is also written in and operates in any operating system where a full copy of can be installed. The output from any analysis function is written into an MS Excel workbook with multiple sheets for the estimates, model statistics, analysis information and the calling syntax itself for reproducing the analysis in future. The flexible design of allows for the quick addition of new studies, analysis types and features to the existing ones.

What Explains Vietnam’s Exceptional Performance in Education Relative to Other Countries? Analysis of the Young Lives Data from Ethiopia, Peru, India and Vietnam

Vietnam’s strong performance on the 2012 and 2015 PISA assessments has led to interest in what explains the strong academic performance of Vietnamese students. Analysis of the PISA data has not shed much light on this issue. This paper analyses a much richer data set, the Young Lives data for Ethiopia, India (Andhra Pradesh and Telangana), Peru and Vietnam, to investigate the reasons for the strong academic performance of 15-year-olds in Vietnam. Differences in observed child and household characteristics explain 37-39% of the gap between Vietnam and Ethiopia, while observed school variables explain only about 3-4 additional percentage points (although an important variable, math teachers’ pedagogical skills, is not available for Ethiopia). Differences in observed child and household characteristics explain very little of the gaps between Vietnam and India and between Vietnam and Peru, yet one observed school variable has a large explanatory effect: primary school math teachers’ pedagogical skills. It explains about 10-12% of the gap between Vietnam and India, raising the overall explained portion to 14-21% of the gap. For Peru, it explains most (65-84%) of the gap.

2018 Australian TALIS-PISA Link Report

Australia was one of nine countries and economies to participate in the 2018 TALIS-PISA link study, together with Cuidad Autónoma de Buenos Aires (Argentina), Colombia, the Czech Republic, Denmark, Georgia, Malta, Turkey and Viet Nam. This study involved coordinating the samples of schools that participated in the Program of International Student Assessment (PISA, a study of the performance of 15-year-old students) and the Teaching and Learning International Survey (TALIS, a study that surveys teachers and principals in lower secondary schools) in 2018. A sample of teachers from schools that were selected to participate in PISA were invited to respond to the TALIS survey. TALIS data provides information regarding the background, beliefs and practices of lower secondary teachers and principals, and PISA data delivers insights into the background characteristics and cognitive and non-cognitive skills of 15-year-old students. Linking these data offers an internationally comparable dataset combining information on key education stakeholders. This report presents results of analyses of the relationships between teacher and school factors and student outcomes, such as performance on the PISA assessment, expectations for further study and experiences of school life. Results for Australia are presented alongside those of the average (mean) across all countries and economies that participated in the TALIS-PISA link study for comparison, but the focus remains on what relationships were significant among Australian students.

2018 Australian TALIS-PISA Link Report

Australia was one of nine countries and economies to participate in the 2018 TALIS-PISA link study, together with Cuidad Autónoma de Buenos Aires (Argentina), Colombia, the Czech Republic, Denmark, Georgia, Malta, Turkey and Viet Nam. This study involved coordinating the samples of schools that participated in the Program of International Student Assessment (PISA, a study of the performance of 15-year-old students) and the Teaching and Learning International Survey (TALIS, a study that surveys teachers and principals in lower secondary schools) in 2018. A sample of teachers from schools that were selected to participate in PISA were invited to respond to the TALIS survey. TALIS data provides information regarding the background, beliefs and practices of lower secondary teachers and principals, and PISA data delivers insights into the background characteristics and cognitive and non-cognitive skills of 15-year-old students. Linking these data offers an internationally comparable dataset combining information on key education stakeholders. This report presents results of analyses of the relationships between teacher and school factors and student outcomes, such as performance on the PISA assessment, expectations for further study and experiences of school life. Results for Australia are presented alongside those of the average (mean) across all countries and economies that participated in the TALIS-PISA link study for comparison, but the focus remains on what relationships were significant among Australian students.

International Comparative Study of Statistics Learning Trajectories Based on PISA Data on Cognitive Diagnostic Models

The cognitive diagnosis model is an emerging evaluation theory. The mastery of fine-grained knowledge points of students can be obtained via the cognitive diagnostic model (CDM), which can subsequently describe the learning trajectory. The latter is a description of the learning progress of students in a specific area, through which teaching and learning can be linked. This research is based on nine statistical items in the Program for International Student Assessment (PISA) 2012 and an analysis of the response data of 30,092 students from 14 countries from four attributes based on CDM. Then, it obtains the learning trajectory of students in statistical knowledge. The study found that Bulgaria, Costa Rica, Peru, Mexico, and Serbia have the same learning trajectories. The learning trajectories of almost 14 countries are as follows: (1) uncertainty, (2) data handling, (3) statistical chart, and (4) average.

ICT Use and Successful Learning: The Role of the Stock of Human Capital

Previous empirical studies have found a weak nexus between the use of information and communication technologies (ICT) for learning and students’ outcomes. However, this literature has not considered the role that the countries’ stock of human capital can have in the successful use of ICT for learning. In this paper, we test empirically the existence of complementarities between human capital and technology adoption for learning. We carry out an empirical analysis with PISA data from a large-scale sample of 363,412 students enrolled in 13,215 schools in 48 countries. We estimate a hierarchical linear model (HLM) of three levels: students, schools, and countries. Our results strongly support the evidence of a positive externality of the stock of human capital on ICT use for learning. When we consider the moderator-effect of the stock of human capital, we find that the negative outcome of ICT use on students’ outcomes in mathematics, reading and science turns positive (greater and more positive the higher the stocks of human capital are). The greater the stock of human capital an economy has, the more benefits it can get from investments in ICT for learning.

Efficiency Analysis with Educational Data: How to Deal with Plausible Values from International Large-Scale Assessments

International large-scale assessments (ILSAs) provide several measures as a representation of educational outcomes, the so-called plausible values, which are frequently interpreted as a representation of the ability range of students. In this paper, we focus on how this information should be incorporated into the estimation of efficiency measures of student or school performance using data envelopment analysis (DEA). Thus far, previous studies that have adopted this approach using data from ILSAs have used only one of the available plausible values or an average of all of them. We propose an approach based on the fuzzy DEA, which allows us to consider the whole distribution of results as a proxy of student abilities. To assess the extent to which our proposal offers similar results to those obtained in previous studies, we provide an empirical example using PISA data from 2015. Our results suggest that the performance measures estimated using the fuzzy DEA approach are strongly correlated with measures calculated using just one plausible value or an average measure. Therefore, we conclude that the studies that decide upon using one of these options do not seem to be making a significant error in their estimates.

About the Equivalence of the Latent D-Scoring Model and the Two-Parameter Logistic Item Response Model

This article shows that the recently proposed latent D-scoring model of Dimitrov is statistically equivalent to the two-parameter logistic item response model. An analytical derivation and a numerical illustration are employed for demonstrating this finding. Hence, estimation techniques for the two-parameter logistic model can be used for estimating the latent D-scoring model. In an empirical example using PISA data, differences of country ranks are investigated when using different metrics for the latent trait. In the example, the choice of the latent trait metric matters for the ranking of countries. Finally, it is argued that an item response model with bounded latent trait values like the latent D-scoring model might have advantages for reporting results in terms of interpretation.

About the Equivalence of the Latent D-Scoring Model and the Two-Parameter Logistic Item Response Model

This article shows that the recently proposed latent D-scoring model of Dimitrov is statistically equivalent to the two-parameter logistic item response model. An analytical derivation and a numerical illustration are employed for demonstrating this finding. Hence, estimation techniques for the two-parameter logistic model can be used for estimating the latent D-scoring model. In an empirical example using PISA data, differences of country ranks are investigated when using different metrics for the latent trait. In the example, the choice of the latent trait metric matters for the ranking of countries. Finally, it is argued that an item response model with bounded latent trait values like the latent D-scoring model might have advantages for reporting results in terms of interpretation.