Re-envisioning assessment of interlanguage pragmatic competence through computer-mediated communicative tasks

Haider, Iftikhar

Permalink

https://hdl.handle.net/2142/105126 Copy

Description

Title

Re-envisioning assessment of interlanguage pragmatic competence through computer-mediated communicative tasks

Author(s)

Haider, Iftikhar

Issue Date

2019-04-19

Director of Research (if dissertation) or Advisor (if thesis)

• Davidson, Fred
• Bowles, Melissa

Doctoral Committee Chair(s)

Sadler, Randall

Committee Member(s)

Roever, Carsten

Department of Study

Linguistics

Discipline

Linguistics

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Language Assessment
• Interlanguage Pragmatics
• Computer-Mediated
• Interactive
• Testing
• Communicative
• ESL

Abstract

Assessment of second language (L2) pragmatic knowledge is still a new and understudied area of research. Some researchers (Hudson, Detmer, & Brown, 1992, 1995; Roever, 2006, Walters, 2007) have played an important role in advancing the field, but their theories have followed speech act framework often criticized for pragmatic construct under-representation (Grabowski, 2009; Roever, 2011; Youn, 2015). Methodologically, past studies mainly used closed role-plays based on predetermined interactional outcomes (Youn, 2015). Kasper and Rose (2002) doubted the validity and authenticity of closed role-play tasks. In order to address these research gaps, and as a contribution to the general understanding of Second Language Pragmatic Testing (SLPT), this study combines second language pragmatics and computer-mediated communication to assess the pragmatic knowledge of second language users of English. It uses Purpura’s (2004) framework of communicative language ability for developing interactive, email-based role-play tasks to assess test takers’ pragmatic ability with regard to sociolinguistic, sociocultural and psychological meanings, and the use of polite formulaic expressions. Using mixed methods (Greene, 2007), qualitative and quantitative evidence was provided to support test inferences. A thorough needs analysis was conducted first through semi-structured interviews and then through an online survey by involving different ESL stakeholders including 153 faculty members at a large Mid-Western university. The online questionnaire results provided insight into the instructors’ perceptions of different email situations. The results of the needs analysis also helped to determine appropriate role-play situations. A set of communicative role-play tasks were developed following Davidson and Lynch’s (2002) test specification theory. Role-play cards were used to enhance standardization, and test takers were allowed to communicate naturally without following fixed interactional outcomes. A group of 52 graduate ESL students completed email role-play tasks. Two native-speaking raters evaluated the pragmatic ability of test takers and assigned scores using an empirically driven analytical scoring rubric on the email threads. Given that most students scored high, there appears to be a correlation between the ESL proficiency level of students and their scores in the pragmatic ability tests. Furthermore, inter-rater reliability analysis shows an overall high inter-rater reliability (0.85). There was some agreement between the hypothesized task difficulty typology and actual scores of three ability groups assessed in the present study. Qualitative analysis of interactive email data revealed a lack of knowledge of norms (as expected in Midwestern US academic settings) of appropriateness and politeness by the lower proficiency groups. Therefore, the low stakes test might have a great potential for developing instructional materials in an academic email communication context. Based on the findings of the present study, suggestions on inclusion of sociopragmatic competence into the ESL writing curriculum are made. Systematic curricular inclusion of email pragmatics in ESL courses will assist ESL learners in developing their email pragmatic competence in academic settings.

Graduation Semester

2019-05

Type of Resource

text

Permalink

http://hdl.handle.net/2142/105126

Copyright and License Information

Copyright 2019 Iftikhar Haider

Owning Collections