A power study of a compromised item detection procedure based on item response theory under different scenarios of subjects’ latent trait

Cao, Canxi

Permalink

https://hdl.handle.net/2142/99525 Copy

Description

Title

A power study of a compromised item detection procedure based on item response theory under different scenarios of subjects’ latent trait

Author(s)

Cao, Canxi

Issue Date

2017-12-12

Director of Research (if dissertation) or Advisor (if thesis)

Zhang, Jinming

Committee Member(s)

• Anderson, Carolyn J.
• Chang, Hua-Hua

Department of Study

Educational Psychology

Discipline

Educational Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Compromised item detection procedure
• Ability with linear growth
• Ability with periodical variation
• Power
• Lag

Abstract

This thesis explores whether or not the changing of students’ latent trait influences the power and the lag performance of a detection procedure based on Item Response Theory for successfully identifying a compromised item in computerized adaptive testing. A simulation study was conducted under three scenarios. The first scenario is the regular scenario, where the students’ latent trait follows the standard normal distribution. In the other two scenarios, the mean of true ability of student population can change in different pattern but not due to the item compromising. Therefore, this simulation mimics two more difficult scenarios, where one shows the ability with linear growth, and the other one has the ability with periodical variation. The simulation experiment yielded five main findings. (1) The mean and median of the distribution of the ability with linear growth scenario were larger than that under the other two scenarios, and the dispersion level of the distribution of the ability with periodical variation scenario is wider than that under the other two conditions; (2) The critical value c_0.01 is always higher than c_0.05, and the value of the moving sample size hardly affects the critical value c_α when moving sample size is greater than 20; (3) Nearly all of the items in the item pool are monitored under all three scenarios conducted in the simulation; (4) The detection procedure always holds a high quality of power (almost stays at 1 all the time); that is, it would not be affected by the changing of students’ latent traits in terms of the power index; (5) The critical values would produce a little bit longer lag under the setting of ability with linear growth scenario than the regular scenario, and there is no difference between the regular scenario and the ability with periodical variation scenario; 6)There is no significant difference of the value of power between α=0.01 and α=0.05, and also for lag.

Graduation Semester

2017-12

Type of Resource

text

Permalink

http://hdl.handle.net/2142/99525

Copyright and License Information

Copyright 2017 Canxi Cao

Owning Collections