Distilling vector space model scores for the assessment of constructed responses with bifactor Inbuilt Rubric method and latent variables

In this paper, we highlight the importance of distilling the computational assessments of constructed responses to validate the indicators/proxies of constructs/trins using an empirical illustration in automated summary evaluation. We present the validation of the Inbuilt Rubric (IR) method that maps rubrics into vector spaces for concepts’ assessment. Specifically, we improved and validated its scores’ performance using latent variables, a common approach in psychometrics. We also validated a new hierarchical vector space, namely a bifactor IR. 205 Spanish undergraduate students produced 615 summaries of three different texts that were evaluated by human raters and different versions of the IR method using latent semantic analysis (LSA). The computational scores were validated using multiple linear regressions and different latent variable models like CFAs or SEMs. Convergent and discriminant validity was found for the IR scores using human rater scores as validity criteria. While this study was conducted in the Spanish language, the proposed scheme is language-independent and applicable to any language. We highlight four main conclusions: (1) Accurate performance can be observed in topic-detection tasks without hundreds/thousands of pre-scored samples required in supervised models. (2) Convergent/discriminant validity can be improved using measurement models for computational scores as they adjust for measurement errors. (3) Nouns embedded in fragments of instructional text can be an affordable alternative to use the IR method. (4) Hierarchical models, like the bifactor IR, can increase the validity of computational assessments evaluating general and specific knowledge in vector space models. R code is provided to apply the classic and bifactor IR method.

The Role of Conditional Likelihoods in Latent Variable Modeling

In psychometrics, the canonical use of conditional likelihoods is for the Rasch model in measurement. Whilst not disputing the utility of conditional likelihoods in measurement, we examine a broader class of problems in psychometrics that can be addressed via conditional likelihoods. Specifically, we consider cluster-level endogeneity where the standard assumption that observed explanatory variables are independent from latent variables is violated. Here, “cluster” refers to the entity characterized by latent variables or random effects, such as individuals in measurement models or schools in multilevel models and “unit” refers to the elementary entity such as an item in measurement. Cluster-level endogeneity problems can arise in a number of settings, including unobserved confounding of causal effects, measurement error, retrospective sampling, informative cluster sizes, missing data, and heteroskedasticity. Severely inconsistent estimation can result if these challenges are ignored.

APPLICATIONS OF ITEM RESPONSE THEORY MODELS TO ASSESS ITEM PROPERTIES AND STUDENTS’ ABILITIES IN DICHOTOMOUS RESPONSES ITEMS

A test is a tool meant to measure the ability level of the students, and how well they can recall the subject matter, but items making up a test may be defectives, and thereby unable to measure students’ ability or traits satisfactorily as intended if proper attention is not paid to item properties such as difficulty, discrimination, and pseudo guessing indices (power) of each item. This could be remedied by item analysis and moderation. It is a known fact that the absence or improper use of item analysis could undermine the integrity of assessment, certification and placement in our educational institutions. Both appropriateness and spread of items properties in accessing students’ abilities distribution, and the adequacy of information provided by dichotomous response items in a compulsory university undergraduate statistics course which was scored dichotomously, and analyzed with stata 16 SE on window 7 were focused here. In view of this, three dichotomous Item Response Theory (IRT) measurement models were used in the context of their potential usefulness in an education setting such as in determining these items properties. Ability, item discrimination, difficulty, and guessing parameters as unobservable characteristics were quantified with a binary response test, then discrete item response becomes an observable outcome variable which is associated with student’s ability level is thereby linked by Item Characteristic Curves that is defined by a set of item parameters that models the probability of observing a given item response by conditioning on a specific ability level. These models were used to assess each of the three items properties together with students’ abilities; then identified defective items that were needed to be discarded, moderated, and non-defectives items as the case may be while some of these chosen items were discussed based on underlining models. Finally, the information provided by these items was also discussed.

Measuring Technologies for CFB Solid Circulation Rate: A Review and Future Perspectives

Solid circulation rate (Gs) represents the mass flux of circulating particles in circulating fluidized bed (CFB) systems and is a significant parameter for the design and operation of CFB reactors. Many measuring technologies for Gs have been proposed, though few of them can be applied in industrial units. This paper presents a comprehensive study on measuring technologies, and the results indicate that though the accumulation method is most widely applied, it is constrained by the disturbance of normal particle circulation. Some publications have proposed mathematic models based on pressure drop or other parameters to establish Gs measurement models; these necessitate the accurate modeling of complicated gas-solid flows in industrial devices. Methods based on certain measurement devices to specify parameters like velocity require device endurance in the industrial operation environment and stable local gas-solid flow. The Gs measuring technologies are strongly influenced by local gas-solid flow states, and the packed bed flow in standpipes make the bottom of standpipes an ideal place to realize Gs measurement.

Risk Stratification and Treatment in Smoldering Multiple Myeloma

Smoldering multiple myeloma is a heterogeneous asymptomatic precursor to multiple myeloma. Since its identification in 1980, risk stratification models have been developed using two main stratification methods: clinical measurement-based and genetics-based. Clinical measurement models can be subdivided in three types: baseline measurements (performed at diagnosis), evolving measurements (performed over time during follow-up appointments), and imaging (for example, magnetic resonance imaging). Genetic approaches include gene expression profiling, DNA/RNA sequencing, and cytogenetics. It is important to accurately distinguish patients with indolent disease from those with aggressive disease, as clinical trials have shown that patients designated as “high-risk of progression” have improved outcomes when treated early. The risk stratification models, and clinical trials are discussed in this review.

2PL Model: Compare Generalized Linear Mixed Model with Latent Variable Model based IRT framework

Fitting item response theory (IRT) models using the generalized mixed logistic regression model (GLMM) has become more popular in large-scale assessment because GLMM allows combining complicated multilevel structures (i.e., students are nested in classrooms which are nested in schools) with IRT measurement models. However, the estimation accuracy of item parameters between these two models is not well examined. This study aimed to compare the estimation results of the GLMM based 2PL model (using the PLmixed R package) with the traditional IRT model (using flexMIRT software) under different sample sizes (N= 500, 1000, 5000) and test length (J = 15, 21) conditions. The simulation results showed that for both the GLMM-based method and the traditional method, item threshold estimates had lower bias than item discrimination parameters. We also found that according to the simulation study, GLMM estimates via PLmixed had lower accuracy than traditional IRT modeling via flexMIRT for items with high discrimination.

Economic Student Scientific Publication Model Exploration Factor Analysis (EFA) Approach

Background: Our research reveals factor measures, which are generated to encourage economics students to engage in scientific publication and research processes. Research purposes: Availability of motivational models for scientific publications, as a form of developing economics students in the future. Research methods: Estimation using Exploratory Factor Analysis (EFA) approach, with statistical tools Jeffreys's Amazing Statistics Program (JASP) version 0.16.0.0. Research results: This study shows the accuracy of the model, including the calculated value of McDonald's and Cronbach's > 0.700, for the MSA value of 0.762, and the value of Bartlett's test < .001. For the correlation relationship, it is strengthened by the RMSEA number which is between 0.05-0.08. This study forms a 3 factor model for scientific publications of economics students. Conclusion: This study estimates the factors that can encourage the scientific publication model of economics students. Several factors in this research model are in line with the findings of previous researchers. This study shows that the accuracy of the model includes the McDonald's and Cronbach's > 0.700, for the MSA value of 0.762, and the Bartlett's test value < .001. For the correlation relationship, it is strengthened by the RMSEA number which is between 0.05-0.08. Thus, there are 3 factors in this model, namely the role of lecturers and families, students' basic abilities, and academic achievement goals. We realize that there are several theoretical challenges and measurement models, therefore further research is carried out using statistical test instruments and tools such as AMOS, PLS, and LISREL.

Adaptation and Validation of a Chinese Online Self-Regulated Learning Questionnaire

This study aimed to adapt and validate a Chinese version of the online self-regulated learning questionnaire (COSLQ) with Chinese junior secondary students in Hong Kong. A total of 716 students from six schools participated voluntarily in the study. Overall, the findings of this study supported the COSLQ’s psychometric quality. The COSLQ subscales all demonstrated high internal consistency. Different measurement models were tested using confirmatory factor analysis. The results indicated that a 7-factor model best fit the data, suggesting that the participants could distinguish seven types of online self-regulatory strategies: goal setting, environment structuring, time management, effort regulation, cognitive/monitoring strategies, help seeking, and self-evaluation.

Price Risk Measurement of China’s Soybean Futures Market Based on the VAR-GJR-GARCH Model

As one of the main forces in the futures market, agricultural product futures occupy an important position in China’s market. As China’s futures market started late and its maturity was low, there are many risks. This study focuses on the Dalian soybean futures market. Dynamic risk measurement models were established to empirically analyze risk measurement problems under different confidence levels. Then, the conditional variance calculated by the volatility model was introduced into the value-at-risk model, and the accuracy of the risk measurement was tested using the failure rate test model. The empirical results show that the risk values calculated by the established models at the 99% and 95% confidence levels are more valuable through the failure rate test, and the risk of China’s soybean futures market can be measured more accurately. The characteristics of “peak thick tail” and “leverage effect” are added to the combination model to calculate the conditional variance more accurately. The failure rate test method is used to test the model, which enriches the research problem of risk measurement.