The Effect of Various Weighting Schemes in The Scoring of Patient Management Problems in Medicine
Bligh, Thomas James
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/65947
Description
Title
The Effect of Various Weighting Schemes in The Scoring of Patient Management Problems in Medicine
Author(s)
Bligh, Thomas James
Issue Date
1980
Department of Study
Education
Discipline
Education
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
Education, Tests and Measurements
Language
eng
Abstract
The primary purpose of this study was to determine if the various weighting schemes which are currently being used to score Patient Management Problem simulations produce scores which are similar enough to be used interchangeably. A second purpose was to identify measurement characteristics which would assist the test constructor in selecting among the various weighting schemes.
Three linear-format PMPs were selected from a previously administered senior medical student certification examination. The subjects were 268 senior medical students from the University of Illinois College of Medicine.
The three most widely used weighting schemes were selected. They were: (1) System A. +1,0,-1; (2) System B. +2,+1,0,-1,-2; (3) System C. +16,+8,+4,+2,+1,0.-1,-2,-4,-8,-16.
Four physicians who had extensive experience with the construction and scoring of PMPs volunteered to participate in the assignment of weights and setting of the standard of acceptable performance (the Minimum Passing Level - MPL).
The order of procedure for the weighting was system A on all three problems, followed by system B on all three problems, followed by system C.
The following conclusions were drawn from the study data: (1) There were very high correlations among the scores obtained from the experimental weighting schemes. (2) There were discrepancies for greater than 10% of the examinees in their pass/fail classification on scores obtained from the experimental weighting schemes when employing the predetermined passing standard (the MPL). However, an analysis of the discrepancies reveals that most were due to differences in the number of failures caused by differences in the strictness of the criterion from one scheme to another. Differences attributable to score order occurred for only 2% of the subjects and it can be concluded that there are no meaningful differences due to score order among the pass/fail decisions made on the basis of the MPL. (3) There were no meaningful discrepancies in pass/fail classifications on scores obtained from the experimental weighting schemes in this study when employing a 1.8% cut-off criterion. (4) The three experimental weighting schemes were found to be approximately equal in Alpha reliability coefficients, correlations with two external criteria, and in interproblem correlations of the schemes.
There were two major questions in this study. The first was: Are the scores obtained by various weighting schemes for written Patient Management Problems similar enough to be used interchangeably? The conclusion is that for the problems and scoring systems examined in this study the scores are similar enough to be used interchangeably. The evidence consists of very high correlations among the scores obtained by the various systems and the small number of discrepancies in pass/fail classifications due to differences in ordering of the scores using either a predetermined standard (the MPL) or a norm referenced standard.
The second question was: Are there qualities of the scores obtained from the various schemes which would indicate a preference for one scheme over the others? The conclusion is that no preference is suggested by the indices which were examined. None of the statistical comparisons revealed any significant differences.
Several areas for future research were identified which included an examination of the effect of large numbers of negative points in a problem, potential problems with the use of the MPL system with other than the weighting system for which it was designed, factors which might aid in the selection of a weighting scheme, and considerations of an entirely different type of scoring system.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
