Station time requirements for performance-based examinations: Their influence on reliability and validity
Shatzer, John Howard, Jr
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/21428
Description
Title
Station time requirements for performance-based examinations: Their influence on reliability and validity
Author(s)
Shatzer, John Howard, Jr
Issue Date
1991
Doctoral Committee Chair(s)
West, Charles K.
Department of Study
Education
Discipline
Education
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Date of Ingest
2011-05-07T13:08:22Z
Keyword(s)
Education, Tests and Measurements
Health Sciences, Education
Education, Educational Psychology
Language
eng
Abstract
Within the past fifteen years there has been increasing interest in using performance-based examinations to supplement or replace written examinations designed to assess clinical competence in medical education. Despite the widespread interest in this type of assessment method, obtaining reliable performance estimates remains a major obstacle in their use. For acceptable estimates usually 12 to 25 medical cases must be included in the examination. With each case taking between 20 and 30 minutes to complete, the administration of a performance-based test is unfeasible for many medical schools.
The purpose of this research was to determine whether an exam with shorter time allocations provides as reliable score estimates as an exam with longer stations. A second purpose was to explore the relationship between station times and the characteristics of the scoring information collected. Fifteen second-year medical students were administered 11 standardized patient cases in a multiple-station format. The length of each station (case) was set at 20 minutes. Scores were analyzed over 5-, 10-, and 20-minute intervals as an indication of performance at different station lengths. Performance at each station was scored by means of a checklist of medical history questions deemed important to ask by the examinee. Checklist elements were organized into one of three categories: elaboration of the chief complaint, associated symptoms, and associated medical history. These categories were used to determine examinee response patterns over the time interval studied. The highest generalizability estimate was obtained during the first 10 minutes of the eleven stations. A second study was conducted with 23 second-year students taking an 8-station examination comprised of cases from the previous test. Station time was set at 10 minutes with scoring at 5- and 10-minute intervals. The highest generalizability was again obtained at 10 minutes. An analysis of response patterns showed that examinees did not change their approach to asking history questions when faced with shorter station time allocations. Results are discussed in terms of their implications for making these examinations more feasible yet retaining their reliability and validity.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
