University of Illinois Urbana-Champaign

Evaluating microblogging sentiments with online workers

Organisciak, Peter

Content Files
organisciak.mp3
organisciak.ppt

Permalink

https://hdl.handle.net/2142/29988

Description

Title
Evaluating microblogging sentiments with online workers
Author(s)
Organisciak, Peter
Issue Date
2011
Keyword(s)
sentiment, microblogging
Date of Ingest
2012-03-12T14:26:01Z
Abstract
The classification of sentiment is a unique challenge for text analysis. Study in the area returns weaker results than in other classification tasks. A reason for this is the complexity of interlinked intra-document sentiments, a problem for not only the effectiveness of mining techniques but also for the annotation of training data. This study explored a correction to this problem by generating a much less complex corpus for sentiment analysis from collected and annotated microblogging messages. The goal of this study was to find the best text mining approach for determining the tone of a sentence-length political sentiment, through the study of a corpus of tweets with a subset classified by humans. This presentation will focus on the results of the first step of the study, the generation of a training corpus. Messages were classified according to sentiment multiple times, independently by anonymous online workers. An iterative ranking algorithm was developed to determine confidence in individual ratings and the workers making them. This showed encouraging improvement in reconciling inter-coder variance over a baseline measure of selection by majority rating. This presentation will also briefly describe the efficacy of classification methods run on the corpus and early findings of a follow-up study measuring the equivalence of ranked online workers to trusted traditional workers.
Publisher
Graduate School of Library and Information Science. University of Illinois at Urbana-Champaign.
Type of Resource
  • other
  • audio
Genre of Resource
Presentation / Lecture / Speech
Language
en
Permalink
http://hdl.handle.net/2142/29988

Owning Collections

Research Showcase 2011 - Graduate School of Information Sciences PRIMARY