Item selection methods in multidimensional computerized adaptive testing adopting polytomously-scored items under multidimensional generalized partial credit model

Lin, Haiyan

Permalink

https://hdl.handle.net/2142/34534 Copy

Description

Title

Item selection methods in multidimensional computerized adaptive testing adopting polytomously-scored items under multidimensional generalized partial credit model

Author(s)

Lin, Haiyan

Issue Date

2012-09-18T21:24:31Z

Director of Research (if dissertation) or Advisor (if thesis)

Chang, Hua-Hua

Doctoral Committee Chair(s)

Ryan, Katherine E.

Committee Member(s)

• Chang, Hua-Hua
• Anderson, Carolyn J.
• Douglas, Jeffrey A.

Department of Study

Educational Psychology

Discipline

Educational Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Multidimensional Computerized Adaptive Testing
• Item Selection Methods
• Multidimensional Item Response Theory
• Multidimensional Generalized Partial Credit Model

Abstract

Four item selection methods are compared and investigated under three test formats in the context of Multidimensional Computerized Adaptive Testing (MCAT) delivering polytomous items partially or completely in tests. Item selection methods examined include Fisher information based D-optimality (D-optimality), Kullback-Leibler information index (KI), mutual information (MI), and continuous entropy method (CEM). The three test formats considered are the POLYTYPE format that contains polytomous items with three response categories, the DPMIX format that delivers dichotomous items at the beginning and polytomous items at the final stage, and the PDMIX format that has the reverse order as DPMIX. In general, D-optimality shows the best estimation accuracy and conditional estimation accuracy. D-optimality, MI, and CEM are similar in terms of ability estimation accuracy and tendency in selecting items when the item bank size is large. For both dichotomous and polytomous items, KI is mostly outperformed by the other three methods in terms of ability estimation precision.When sub-thetas in both dimensions are equal,however,KI shows the best performance for polytomous items. In this study, which item type, dichotomous or polytomous, being administered first does not affect the estimation accuracy. However, if the test length is much longer or shorter than the test length of the current study, it is possible that the estimation accuracy could be affected by the order of delivering different item types. Both DPMIX and PDMIX formats yield similar conditional estimation accuracy pattern and precision. In addition, the item bank size does affect the estimation precision. These conclusions, however, might not be applied to MCAT testing with different test designs or item pool structures. More studies are needed in MCAT combining with polytomous items to further facilitate the development and improvement of the next-generation assessments such as formative assessment or testing for diagnosis.

Graduation Semester

2012-08

Permalink

http://hdl.handle.net/2142/34534

Copyright and License Information

Copyright 2012 Haiyan Lin

Owning Collections