University of Illinois Urbana-Champaign

New methods of online calibration for item bank replenishment

Zheng, Yi

Content Files
Yi_Zheng.pdf

Permalink

https://hdl.handle.net/2142/50699

Description

Title
New methods of online calibration for item bank replenishment
Author(s)
Zheng, Yi
Issue Date
2014-09-16
Director of Research (if dissertation) or Advisor (if thesis)
Chang, Hua-Hua
Doctoral Committee Chair(s)
Chang, Hua-Hua
Committee Member(s)
  • Anderson, Carolyn J.
  • Culpepper, Steven A.
  • Douglas, Jeffrey A.
  • Ryan, Katherine E.
  • Zhang, Jinming
Department of Study
Educational Psychology
Discipline
Educational Psychology
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Date of Ingest
2014-09-16T17:25:23Z
Keyword(s)
  • online calibration
  • computerized adaptive testing
  • item calibration
  • sequential optimal design
  • item response theory
Abstract
"Item parameter calibration is important for tests based on item response theory, because the scoring, equating, bias analysis of the test, and item selection in adaptive tests are all based on the item parameters. As a test is continuously administered, item calibration needs to be conducted for new items at intervals to replace overexposed, obsolete, or flawed items in the item bank. Although it is possible to recruit examinees for the sole purpose of pretesting the new items, a more cost-effective and commonly employed approach is to embed the new items in operational tests. When this approach is employed in computerized adaptive tests (CAT), it is called ""online calibration."" Analogous to the tailored testing feature in CAT, where an optimal set of operational items are selected for each examinee to more efficiently estimate their ability levels, online calibration makes it possible to select an optimal sample of examinees for each pretest item to more efficiently calibrate their item parameters. During the operational tests, different pretest items can be selected for each examinee. The parameter values of the pretest items are constantly updated, based on which the sampling scheme is dynamically adjusted. A few pretest item selection methods have been proposed, but such development is still in its infant phase. This thesis proposes a new framework for pretest item selection in online calibration. A simulation study was conducted to compare the proposed methods with existing methods and also compare different estimation methods and pretest item seeding locations. Results show significant superiority of the proposed methods compared to existing methods in the 1PL and 2PL models. Middle and late seeding locations lead to more accurate calibration results. The Bayesian MEM estimation method is recommended among the six compared estimation methods."
Graduation Semester
2014-08
Permalink
http://hdl.handle.net/2142/50699
Copyright and License Information
Copyright 2014 Yi Zheng

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Education
Dissertations and Theses from the College of Education