An investigation of the cognitive foundation underlying the rule-space model

Gierl, Mark John

Permalink

https://hdl.handle.net/2142/21041 Copy

Description

Title

An investigation of the cognitive foundation underlying the rule-space model

Author(s)

Gierl, Mark John

Issue Date

1996

Doctoral Committee Chair(s)

Harnisch, Delwyn L.

Department of Study

Education

Discipline

Education

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Education, Tests and Measurements
• Education, Educational Psychology
• Psychology, Cognitive

Language

eng

Abstract

• The purpose of this study is to combine the theories and methods in cognitive psychology with the practices in psychometrics to investigate the cognitive foundations underlying K. Tatsuoka's rule-space model. Rule space is a statistical method for classifying students' test item responses into a set of attribute mastery patterns associated with different cognitive skills. An attribute is a description of the procedures, skills, processes, strategies, and knowledge a student must possess to solve a test item. The attributes for a test are identified by test developers and are intended to reflect students' problem-solving thought processes. Therefore, it is appropriate to compare these two representations of thinking as a way of understanding the psychology of complex test performance.
• Thirty high school students (16 males; 14 female) were asked to think aloud as they solved 16 algebra items taken from the November 1994 administration of the Scholastic Achievement Test (SAT). Flow charts were then created from the student think aloud protocols and compared to the item attributes identified by K. Tatsuoka and her colleagues to evaluate the veracity of the attribute descriptions.
• The observed attribute structure derived from the student flow charts matched the expected attribute structure identified by K. Tatsuoka and her associates for 95% of the cases studied on the 16-item SAT algebra subtest indicating that the attributes contained an accurate description of the cognitive skills used by examinees. However, a more complex picture emerged when each strategy and solution path was considered. Three additional findings were reported. First, eight of the 16 items on the SAT algebra subtest contained strategy-dependent attributes. Strategy-dependent attributes result whenever the structure of the attributes for an item varies with the strategy used by students to solve that item. Second, attributes 12, 14, and 15 may not provide useful information about the cognitive skills needed to solve items on the SAT. The cognitive skills associated with attribute 12, test-taking skills, were rarely used by students on the SAT algebra subtest. The cognitive skills associated with attribute 14, quantitative and logical reading, and attribute 15, open-ended and grid-ins, measured specific skills related to formatting or filling-in a final solution on the SAT score sheet. Third, the attribute structure for some of the test items was too broad and encompassing. As a result, the expected attribute structure contained cognitive skills that were both relevant and irrelevant to the strategies and solution paths used by students to solve the problems.
• The rule-space model is a psychometric procedure for identifying students' cognitive proficiencies. However, the success of the model in linking cognitive and psychometric theory depends on how well attributes represent the cognitive problem-solving skills students actually use to solve test items. The purpose of the present study is to begin to evaluate the assumptions about cognition inherent in the rule-space model.

Type of Resource

text

Permalink

http://hdl.handle.net/2142/21041

Copyright and License Information

Copyright 1996 Gierl, Mark John

Owning Collections