Interest item diagnostics through IRT: an application of GPCM and tree-based IRT models to contemporary measures of interests

Wee, Jian Ming Colin

Permalink

https://hdl.handle.net/2142/110540 Copy

Description

Title

Interest item diagnostics through IRT: an application of GPCM and tree-based IRT models to contemporary measures of interests

Author(s)

Wee, Jian Ming Colin

Issue Date

2021-04-23

Director of Research (if dissertation) or Advisor (if thesis)

Rounds, James B

Doctoral Committee Chair(s)

Rounds, James B

Committee Member(s)

• Newman, Daniel A
• Kern, Justin L
• Napolitano, Christopher M
• Hoff, Kevin A

Department of Study

Psychology

Discipline

Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• vocational interests
• item response theory
• psychometrics

Abstract

Item Response Theory (IRT) is a contemporary test evaluation technique that is frequently used in the fields of achievement and aptitude testing. However, IRT research within the field of vocational interest test evaluation is nascent at best. This dissertation advances the use of IRT in vocational interest through two studies of contemporary interest measures: The O*NET Interest Profiler (Rounds, Hoff & Lewis, 2021) and the Comprehensive Assessment of Basic Interests (CABIN; Su, Tay, Liao, Zhang & Rounds, 2018). For the O*NET Interest Profiler (IP), I use IRT to demonstrate the viability of a new tree-based response model for answering inventory items and compare the results of test shortening using the tree-based IRT model versus a more standard 2 Parameter Logistic Model. I found that the tree based model was more apt for describing the ‘Like/Dislike/Unsure’ response data of the O*NET IP and discovered that respondents tend to form similar thought processes when approaching these interest items—They make a two-step decision of whether or not they like the item first, before deciding if they actually dislike the item second. For the CABIN, I use a Generalized Partial Credit Model (GPCM) to obtain item diagnostics for the 8 interest subscales and propose a shortened version of each scale using item information curves and marginal reliability coefficients. I proposed two short forms of the CABIN based on these diagnostics—One that preserves the 41 basic interest scales of the original, and an even shorter version that simply measures the 8 overarching interest constructs. Researchers should select a short form that matches their desired level of specificity of the measured construct, and I illustrate how IRT can help in this endeavor. Both studies reveal the untapped potential IRT has for refining measures in vocational interest research and understanding the item response process of interest items.

Graduation Semester

2021-05

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/110540

Copyright and License Information

Copyright 2021 Colin Wee Jian Ming

Owning Collections