The effect of item pool size and stopping rule on bias and error parameters in computerized adaptive testing

OpenAlex · English

This study aims to examine the effects of item pool size and stopping rules on bias and error parameters in computerized adaptive testing (CAT). Simulated data were generated and analyzed for item pools of 200 and 2000 items, fixed-length and error stopping rules, and eight different combinations of Maximum Likelihood (ML) and Expected a Posteriori (EAP) methods for ability estimation. For each sub-problem, the findings are listed in a general table and a collection of graphs. In the light of the findings, the error and bias parameters obtained with the ML skill estimation method with fixed-length stopping rule in both small and large item pools were found to be lower than the EAP method. Similarly, in both small and large item pools, the error and bias parameters obtained with the ML ability estimation method in which the error stopping rule was applied were found to be lower than the EAP method. While the number of items used in the analyses where the error stopping rule was applied was approximately six in both ability estimation methods for the small item pool, approximately five items were used with the EAP method and approximately 21 items were used with the ML method in the large item pool. The application with the lowest RMSE and bias values and the highest correlation between actual and predicted θ values was the application related to the sixth sub-problem in which a large item pool was used, a fixed-length stopping rule was used, and ability estimation was performed with the ML method. The results of the study provide clues for the development of testing strategies in CAT applications and provide guidance for more effective assessment processes in the field of education.