Evaluating the robustness of FlexMIRT on DIF analysis

Liu, Yiqing

Permalink

https://hdl.handle.net/2142/122179 Copy

Description

Title

Evaluating the robustness of FlexMIRT on DIF analysis

Author(s)

Liu, Yiqing

Issue Date

2023-12-07

Director of Research (if dissertation) or Advisor (if thesis)

Zhang, Jinming

Committee Member(s)

• Xia, Yan
• Zhang, Susu

Department of Study

Educational Psychology

Discipline

Educational Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

Item Response Theory, Differential Item Functioning, flexMIRT

Abstract

Item Response Theory (IRT) plays a crucial role in educational measurement, and accurately detecting Differential Item Functioning (DIF) items is critical to ensuring fairness and validity in tests. This study focused on flexMIRT, a software for multilevel and IRT analysis, performance in DIF analysis within the three-parameter logistic (3PL) and four-parameter logistic (4PL) models. Specifically, the aim was to evaluate the robustness of flexMIRT when applied to the 4PL model, which was not the software's true model. Simulation studies evaluated the ability to control Type I error rates and detect DIF items with varying sample sizes, test lengths, percentages of DIF items, and different levels of parameter change. The results of the simulation studies indicated that flexMIRT maintains low Type I error rates for discrimination (a) and difficulty (b) parameters across both models. An uptick in Type I error rates for the test of guessing parameter (g) was noted as the sample size increased. For Type II error rates, flexMIRT demonstrated enhanced detection capabilities in larger samples and with significant parameter changes, affirming its effectiveness in different IRT models. While flexMIRT showed adaptability in DIF detection, careful interpretation was warranted for the g parameter in extensive datasets. Future research directions include evaluating flexMIRT's robustness with IRT models, exploring a more comprehensive array of testing conditions, and establishing a more definitive framework for detecting significant DIF to provide precise usage guidelines for flexMIRT.

Graduation Semester

2023-12

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/122179

Copyright and License Information

Copyright 2023 Yiqing Liu

Owning Collections