Collaboration in Language Testing: Investigating the Validity of a New Rating Procedure for EFL Writing Assessment in Korea
Lee, Ho
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/79839
Description
Title
Collaboration in Language Testing: Investigating the Validity of a New Rating Procedure for EFL Writing Assessment in Korea
Author(s)
Lee, Ho
Issue Date
2005
Doctoral Committee Chair(s)
Davidson, Fred
Department of Study
Education
Discipline
Education
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
Education, Language and Literature
Language
eng
Abstract
This study seeks an innovative testing system in which trained students participate in a writing test as a complementary rater, specifically in the EFL context. The study has three research goals: (1) to investigate the psychometric reliability of self- and peer-ratings in a second language (L2) essay-writing test in Korea, (2) to explore students' rating process, and (3) to understand student's perception of rating tasks. The rationale of this research is that the Korean EFL students whose major is English education should conduct the rating tasks that they will encounter in their future career. Each of 104 Korean EFL learners at the intermediate level sequentially rated three sample essays, his/her own essay, and his/her partner's essay after he/she was provided with the same benchmark as two experts would refer to. Each essay was rated with a 5-scale in terms of 4 writing features (Organization, Content, Language Use, and Holistic feature). After students finished the rating tasks, they filled out a questionnaire. The students' ratings then were compared to expert ratings, using Multi-facet Rasch model. The quantitative research was accompanied by the qualitative study. Eleven students participated in semi-structured interviews. For the internal and external validity of the research, the researcher took use of three types of triangulations: data triangulation, methodology triangulation, and investigator triangulation. Results showed that peer raters were so lenient that they showed unpredictable rating patterns. From the qualitative analyses, students actively utilized a variety of rating strategies. In addition, they showed the positive attitudes toward the rating tasks. Most interviewees, however, viewed that they were not able to be an expert rater because of their linguistic incompetence. It was concluded that the collaborative rating procedure fosters students to activate high-mental function and a social skills, raising students' critical awareness of their social context.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
