An Empirical, Evaluative Comparison of the Goal-Based and Goal-Free Approaches to Educational Program Evaluation
Walsh, Patrick Leo
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/66022
Description
Title
An Empirical, Evaluative Comparison of the Goal-Based and Goal-Free Approaches to Educational Program Evaluation
Author(s)
Walsh, Patrick Leo
Issue Date
1980
Department of Study
Education
Discipline
Education
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
Education, Educational Psychology
Language
eng
Abstract
Theoretically, the use of different evaluation approaches yields different information. This information often leads to different conclusions, but it is not necessarily inaccurate. It is different information, sometimes relevant, sometimes irrelevant, depending on the audience. A comparative research study of evaluation approaches might provide data on which to base more reasoned decisions concerning which approach to select in a particular evaluation context.
This study chose two outcome-oriented approaches to educational program evaluation (i.e., goal-based and goal-free) and implemented them concurrently on a short-term continuing education workshop for health care professionals. The purpose of the inquiry was to empirically compare in an evaluative mode the goal-based and goal-free approaches to evaluation.
The methodology for the evaluative comparison was modeled after a process described by Taylor (1961). The process entailed developing standard dimensions, each of which was elaborated into operational statements. The standards and their associated statements represented criteria for high quality evaluation. The standards used in the study were adopted from Grotelueschen et al. (1974): purpose, audiences, issues, resources, evidence, data-gathering, analysis, and reporting.
The study produced two types of data: (a) a narrative description or verbatim reproduction of what took place during the implementation of each evaluation approach related to each of the standard dimensions and its associated operational statements, and (b) a verbal and numerical rating indicating the degree to which each evaluation approach fulfilled or failed to fulfill the standard and its operational statements.
The goal-based and goal-free approaches were found to differ substantively on virtually every dimension of comparison. Differences were found in: the types of value considered by each approach (goal-based, instrumental; goal-free, instrumental, contributive, and intrinsic), the role of the evaluator (goal-based, technician; goal-free, archeologist/anthropologist), the resources required to implement each (different expertise demanded of the evaluator, more time for goal-free, goal-free also required a person to act as a screen to filter goal/objective information from the evaluator), the types of evidence considered, the methods by which data were gathered and analyzed, as well as the manner in which the results were reported.
The goal-based approach satisfied the requirements of the evaluation task more successfully than did the goal-free approach. This outcome was attributed to the internal consistency found in the goal-based approach with regard to the purpose, issues, evidence, data-gathering, analysis, and reporting of the evaluation, and the match between what transpired and what was needed. Such consistency and match was not found in the goal-free approach.
The goal-free approach was found to excel in surfacing issues to consider in an evaluation and also in providing insights regarding perspectives which might be taken in viewing a program. The methodology used to compare the two approaches was found deficient in a number of areas. Suggestions were made to improve each shortcoming.
The similarity between Stufflebeam's (1974) definition of meta-evaluation as a "procedure for describing an evaluation activity and judging it against a set of ideas concerning what constitutes good evaluation," and the current study's method was discussed.
Finally, a number of uses of the method for future studies were suggested. These uses ranged from identifying the critical attributes of various evaluation approaches to utilizing the criteria/standards developed for the study as a prescriptive model for performing evaluations.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
