Optimizing the wisdom of the crowd: Learning, teaching, and recommendation

Zhou, Yao

Permalink

https://hdl.handle.net/2142/115353 Copy

Description

Title

Optimizing the wisdom of the crowd: Learning, teaching, and recommendation

Author(s)

Zhou, Yao

Issue Date

2022-04-06

Director of Research (if dissertation) or Advisor (if thesis)

He, Jingrui

Doctoral Committee Chair(s)

He, Jingrui

Committee Member(s)

• Zhai, Chengxiang
• Li, Bo
• Ying, Lei

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Crowdsourcing
• Wisdom of crowd
• Truth Inference, Heterogeneous Learning
• Machine Teaching
• Recommender Systems

Abstract

Crowdsourcing is one type of sourcing model that helps individuals and organizations obtain large amount of information, e.g., ideas, knowledge, financial resources, etc., from a group of non-experts within limited time at a low cost. In recent decades, the unprecedented amount of data has catalyzed the trend of combining human insights with machine learning techniques, and thereby, facilitated the wide usage of crowdsourcing to enlist label information effectively and efficiently. Specifically, the adoption of crowdsourcing has been witnessed in a wide range of high-impact real-world applications, such as collaborative knowledge (e.g., data annotation, language translation), collective creativity (e.g., analogy mining, crowdfunding), and reverse Turing test (e.g., CAPTCHA-like systems), etc. Researchers and practitioners have mainly worked on the collaborative information aggregation problem where the data items are outsourced to a group of unskilled online workers for labeling. Despite the wide adoption of crowdsourcing services, several of its fundamental problems remain unsolved especially at the information and cognitive levels in terms of incentive design, aggregation methodology, and model learning. Specifically, the related computational challenges fall into the following categories of problems: (P1) Truth inference - How to infer the ground truth label of each item that has been distributed and labeled by multiple crowdsourcing labelers? What is the benefit of using truth inference for the downstream learning tasks? (P2) Heterogeneous learning - How to perform a unified model learning when the labeled data demonstrate multiple types of heterogeneity, such as oracle heterogeneity (more than one labels are collected from different labelers per item), task heterogeneity (multiple similar predictive tasks share commonality and are jointly learned), and view heterogeneity (items are characterized by different sources of features)? (P3) Crowd teaching - How to improve the labeling expertise of the crowdsourcing workers in the form of teaching and as a response, benefit the model learning eventually? The teaching mechanism should be personalized, effective, and interpretable. (P4) Recommendation - How to rank the user relevance over the unseen items if only a small number of user’s ratings (instead of the class labels in crowdsourcing) over items are given? How to properly handle the data sparsity, imbalanced sampling, and provide explanations to justify the recommendations? In this thesis, an in-depth study of these four problems is properly motivated, rigorously formulated, thoroughly evaluated, and broadly discussed. Specifically, one tensor augmentation and completion model is proposed to tackle the problem of Truth Inference; two unified optimization frameworks are proposed to solve the Heterogeneous Learning problem with crowdsourced labels that have various types of data heterogeneities - task heterogeneity and view heterogeneity; two Crowd Teaching mechanisms are proposed to guide crowd workers toward the personalized teaching as well as the interpretable concept learning. In the end, we tackle the Recommendation problem by proposing one unified ranking model that combines positive-unlabeled learning with generative adversarial networks to mitigate the data sparsity in collaborative filtering and also one systematical study regarding the explainability of the contextualized recommender systems. Optimizing the wisdom of the crowd is a promising direction with lots of potentials. Towards the end of the thesis, we have also presented a few promising future research directions such as complementary recommendations, fairness recommendations, and federated learning.

Graduation Semester

2022-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Yao Zhou

Owning Collections