Evaluating microblogging sentiments with online workers

Organisciak, Peter

Permalink

https://hdl.handle.net/2142/29988 Copy

Description

Title

Evaluating microblogging sentiments with online workers

Author(s)

Organisciak, Peter

Issue Date

2011

Keyword(s)

sentiment, microblogging

Abstract

The classification of sentiment is a unique challenge for text analysis. Study in the area returns weaker results than in other classification tasks. A reason for this is the complexity of interlinked intra-document sentiments, a problem for not only the effectiveness of mining techniques but also for the annotation of training data. This study explored a correction to this problem by generating a much less complex corpus for sentiment analysis from collected and annotated microblogging messages. The goal of this study was to find the best text mining approach for determining the tone of a sentence-length political sentiment, through the study of a corpus of tweets with a subset classified by humans. This presentation will focus on the results of the first step of the study, the generation of a training corpus. Messages were classified according to sentiment multiple times, independently by anonymous online workers. An iterative ranking algorithm was developed to determine confidence in individual ratings and the workers making them. This showed encouraging improvement in reconciling inter-coder variance over a baseline measure of selection by majority rating. This presentation will also briefly describe the efficacy of classification methods run on the corpus and early findings of a follow-up study measuring the equivalence of ranked online workers to trusted traditional workers.

Publisher

Graduate School of Library and Information Science. University of Illinois at Urbana-Champaign.

Type of Resource

• other
• audio

Language

en

Permalink

http://hdl.handle.net/2142/29988

Owning Collections