Controlling Item Exposure and Test Overlap in Computerized Adaptive Testing

2005 ◽  
Vol 29 (3) ◽  
pp. 204-217 ◽  
Author(s):  
Shu-Ying Chen ◽  
Pui-Wa Lei
Keyword(s):  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Item Exposure ◽  
Test Overlap

Evaluating a Computerized Adaptive Testing Version of a Cognitive Ability Test Using a Simulation Study

2021 ◽  
pp. 073428292110277
Author(s):  
Ioannis Tsaousis ◽  
Georgios D. Sideridis ◽  
Hannan M. AlGhamdi
Keyword(s):  
Cognitive Ability ◽  
Simulation Study ◽  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Estimation Methods ◽  
Item Pool ◽  
Sequential Approach ◽  
Ability Test ◽  
Promising Alternative ◽  
Item Exposure

This study evaluated the psychometric quality of a computerized adaptive testing (CAT) version of the general cognitive ability test (GCAT), using a simulation study protocol put forth by Han, K. T. (2018a). For the needs of the analysis, three different sets of items were generated, providing an item pool of 165 items. Before evaluating the efficiency of the GCAT, all items in the final item pool were linked (equated), following a sequential approach. Data were generated using a standard normal for 10,000 virtual individuals ( M = 0 and SD = 1). Using the measure’s 165-item bank, the ability value (θ) for each participant was estimated. maximum Fisher information (MFI) and maximum likelihood estimation with fences (MLEF) were used as item selection and score estimation methods, respectively. For item exposure control, the fade away method (FAM) was preferred. The termination criterion involved a minimum SE ≤ 0.33. The study revealed that the average number of items administered for 10,000 participants was 15. Moreover, the precision level in estimating the participant’s ability score was very high, as demonstrated by the CBIAS, CMAE, and CRMSE). It is concluded that the CAT version of the test is a promising alternative to administering the corresponding full-length measure since it reduces the number of administered items, prevents high rates of item exposure, and provides accurate scores with minimum measurement error.

scholarly journals Some Alternatives to Sympson-Hetter Item-Exposure Control in Computerized Adaptive Testing

2003 ◽  
Vol 28 (3) ◽  
pp. 249-265 ◽  
Author(s):  
Wim J. van der Linden
Keyword(s):  
Iterative Process ◽  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Law School ◽  
Exposure Control ◽  
Item Exposure ◽  
Item Exposure Control ◽  
Admission Test ◽  
Formal Properties ◽  
Ability Parameter

The Hetter and Sympson (1997 ; 1985 ) method is a method of probabilistic item-exposure control in computerized adaptive testing. Setting its control parameters to admissible values requires an iterative process of computer simulations that has been found to be time consuming, particularly if the parameters have to be set conditional on a realistic set of values for the examinees’ ability parameter. Formal properties of the method are identified that help us explain why this iterative process can be slow and does not guarantee admissibility. In addition, some alternatives to the SH method are introduced. The behavior of these alternatives was estimated for an adaptive test from an item pool from the Law School Admission Test (LSAT). Two of the alternatives showed attractive behavior and converged smoothly to admissibility for all items in a relatively small number of iteration steps.

scholarly journals A Dynamic Stratification Method for Improving Trait Estimation in Computerized Adaptive Testing Under Item Exposure Control

2019 ◽  
Vol 44 (3) ◽  
pp. 182-196
Author(s):  
Jyun-Hong Chen ◽  
Hsiu-Yi Chao ◽  
Shu-Ying Chen
Keyword(s):  
Computerized Adaptive Testing ◽  
Item Difficulty ◽  
Adaptive Testing ◽  
Item Selection ◽  
Exposure Control ◽  
Item Exposure ◽  
Stratification Method ◽  
High Discrimination ◽  
Item Exposure Control ◽  
Trait Estimation

When computerized adaptive testing (CAT) is under stringent item exposure control, the precision of trait estimation will substantially decrease. A new item selection method, the dynamic Stratification method based on Dominance Curves (SDC), which is aimed at improving trait estimation, is proposed to mitigate this problem. The objective function of the SDC in item selection is to maximize the sum of test information for all examinees rather than maximizing item information for individual examinees at a single-item administration, as in conventional CAT. To achieve this objective, the SDC uses dominance curves to stratify an item pool into strata with the number being equal to the test length to precisely and accurately increase the quality of the administered items as the test progresses, reducing the likelihood that a high-discrimination item will be administered to an examinee whose ability is not close to the item difficulty. Furthermore, the SDC incorporates a dynamic process for on-the-fly item–stratum adjustment to optimize the use of quality items. Simulation studies were conducted to investigate the performance of the SDC in CAT under item exposure control at different levels of severity. According to the results, the SDC can efficiently improve trait estimation in CAT through greater precision and more accurate trait estimation than those generated by other methods (e.g., the maximum Fisher information method) in most conditions.

Sign in / Sign up

Export Citation Format

Share Document