Tasker: Safely Serving Verifiable Micro-tasks for Researchers

Jeffrey, Monique; Whiting, Mark; Kozolan, Abrere S.; Motorin, Imauri; Samuel, Osei-Afriyie

Permalink

https://hdl.handle.net/2142/100240 Copy

Description

Title

Tasker: Safely Serving Verifiable Micro-tasks for Researchers

Author(s)

• Jeffrey, Monique
• Whiting, Mark
• Kozolan, Abrere S.
• Motorin, Imauri
• Samuel, Osei-Afriyie

Issue Date

2018

Keyword(s)

• crowdsourcing
• Amazon Mechanical Turk
• research tools

Abstract

Paid crowdsourcing removes many traditional boundaries in conducting participant based research, however with this new tool, new instrumentation challenges have arisen for researchers. Three common challenges include: the difficulty in creating large numbers of high quality and novel tasks, verifying results of the tasks without relying on manual cheat mitigation techniques, and ensuring that the tasks adhere to the latest visual and instructional design to get high quality results. These circumstances endanger current and future research on Amazon Mechanical Turk and can result in compromised data. We introduce Tasker, a secure system architecture for serving unique tasks supported by usability principles to workers, and providing verification information concerning their completion and accuracy to researchers. This poster discusses insights from our pilot study and explorations toward methods that demonstrate a marked improvement for speed, security and robustness in developing tasks for research leveraging Amazon Mechanical Turk.

Publisher

iSchools

Series/Report Name or Number

iConference 2018 Proceedings

Type of Resource

text

Language

eng

Permalink

http://hdl.handle.net/2142/100240

Copyright and License Information

Copyright 2018 is held by Monique Jeffrey, Mark Whiting, Abrere Sophie Kozolan, Imauri Motorin, Osei-Afriyie Samuel. Copyright permissions, when appropriate, must be obtained directly from the authors.

Owning Collections