University of Illinois Urbana-Champaign

Item selection methods in multidimensional computerized adaptive testing adopting polytomously-scored items under multidimensional generalized partial credit model

Lin, Haiyan

Content Files
LIN_HAIYAN.pdf

Permalink

https://hdl.handle.net/2142/34534

Description

Title
Item selection methods in multidimensional computerized adaptive testing adopting polytomously-scored items under multidimensional generalized partial credit model
Author(s)
Lin, Haiyan
Issue Date
2012-09-18T21:24:31Z
Director of Research (if dissertation) or Advisor (if thesis)
Chang, Hua-Hua
Doctoral Committee Chair(s)
Ryan, Katherine E.
Committee Member(s)
  • Chang, Hua-Hua
  • Anderson, Carolyn J.
  • Douglas, Jeffrey A.
Department of Study
Educational Psychology
Discipline
Educational Psychology
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Date of Ingest
2012-09-18T21:24:31Z
Keyword(s)
  • Multidimensional Computerized Adaptive Testing
  • Item Selection Methods
  • Multidimensional Item Response Theory
  • Multidimensional Generalized Partial Credit Model
Abstract
Four item selection methods are compared and investigated under three test formats in the context of Multidimensional Computerized Adaptive Testing (MCAT) delivering polytomous items partially or completely in tests. Item selection methods examined include Fisher information based D-optimality (D-optimality), Kullback-Leibler information index (KI), mutual information (MI), and continuous entropy method (CEM). The three test formats considered are the POLYTYPE format that contains polytomous items with three response categories, the DPMIX format that delivers dichotomous items at the beginning and polytomous items at the final stage, and the PDMIX format that has the reverse order as DPMIX. In general, D-optimality shows the best estimation accuracy and conditional estimation accuracy. D-optimality, MI, and CEM are similar in terms of ability estimation accuracy and tendency in selecting items when the item bank size is large. For both dichotomous and polytomous items, KI is mostly outperformed by the other three methods in terms of ability estimation precision.When sub-thetas in both dimensions are equal,however,KI shows the best performance for polytomous items. In this study, which item type, dichotomous or polytomous, being administered first does not affect the estimation accuracy. However, if the test length is much longer or shorter than the test length of the current study, it is possible that the estimation accuracy could be affected by the order of delivering different item types. Both DPMIX and PDMIX formats yield similar conditional estimation accuracy pattern and precision. In addition, the item bank size does affect the estimation precision. These conclusions, however, might not be applied to MCAT testing with different test designs or item pool structures. More studies are needed in MCAT combining with polytomous items to further facilitate the development and improvement of the next-generation assessments such as formative assessment or testing for diagnosis.
Graduation Semester
2012-08
Permalink
http://hdl.handle.net/2142/34534
Copyright and License Information
Copyright 2012 Haiyan Lin

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Education
Dissertations and Theses from the College of Education