University of Illinois Urbana-Champaign

An approach to the evaluation of expert systems for the selection of reference sources

Su, Shiao-Feng

This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.

Permalink

https://hdl.handle.net/2142/23247

Description

Title
An approach to the evaluation of expert systems for the selection of reference sources
Author(s)
Su, Shiao-Feng
Issue Date
1994
Doctoral Committee Chair(s)
Lancaster, F.W.
Committee Member(s)
  • Smith, Linda C.
  • Allen, Bryce L.
Department of Study
Library and Information Science
Discipline
Library and Information Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • Library Science
  • Information Science
  • Artificial Intelligence
Language
eng
Abstract
  • Sixty students at the University of Illinois voluntarily participated in the research to select reference sources for ten test questions with or without the aid of the expert systems to be evaluated. The students were first divided into little-experienced and no-experienced groups; the students in each group were further randomly divided into RE, SF, and NO groups. The RE group students consulted Reference Expert to select reference sources, the SF group students consulted SourceFinder, and the NO group students consulted the online catalog (IO$\sp+$/IBIS), browsed shelves in the Undergraduate Library, or retrieved from personal knowledge.
  • "The results of the study suggest that the current expert systems for the selection of reference sources cannot perform as well as experienced subject-oriented reference librarians. The better system, Reference Expert, could only achieve 80 percent accuracy in aiding users to select appropriate sources. SourceFinder achieved 40 percent accuracy. A failure analysis is provided. The results indicate that the students without ""expert system"" aid can perform as well as comparable students aided by SourceFinder in the selection of sources. SourceFinder can lead its users to select reference sources at a 25% consistency level, while Reference Expert achieves a 57.5% consistency level. Students not aided by any expert system achieve only 10% consistency in source selection."
  • "The results of this research generally do not offer strong support for the belief that ""expert systems"" for selection of reference sources can provide much help to increase the accuracy of question answering in reference services. In fact, in the case of SourceFinder, students of little experience aided by the system performed significantly worse than did the comparable students using the online catalog or personal knowledge. It is not just a matter of ""why bother"" to build an expert system of this type; there also exists the possibility that exclusive use of such a system to select reference sources may worsen the existing ""half-right"" performance of reference librarians."
Type of Resource
text
Permalink
http://hdl.handle.net/2142/23247
Copyright and License Information
Copyright 1994 Su, Shiao-Feng

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Information Sciences
Dissertations and theses from the School of Information Sciences