University of Illinois Urbana-Champaign

Optimizing the wisdom of the crowd: Learning, teaching, and recommendation

Zhou, Yao

Content Files
ZHOU-DISSERTATION-2022.pdf

Permalink

https://hdl.handle.net/2142/115353

Description

Title
Optimizing the wisdom of the crowd: Learning, teaching, and recommendation
Author(s)
Zhou, Yao
Issue Date
2022-04-06
Director of Research (if dissertation) or Advisor (if thesis)
He, Jingrui
Doctoral Committee Chair(s)
He, Jingrui
Committee Member(s)
  • Zhai, Chengxiang
  • Li, Bo
  • Ying, Lei
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • Crowdsourcing
  • Wisdom of crowd
  • Truth Inference, Heterogeneous Learning
  • Machine Teaching
  • Recommender Systems
Abstract
Crowdsourcing is one type of sourcing model that helps individuals and organizations obtain large amount of information, e.g., ideas, knowledge, financial resources, etc., from a group of non-experts within limited time at a low cost. In recent decades, the unprecedented amount of data has catalyzed the trend of combining human insights with machine learning techniques, and thereby, facilitated the wide usage of crowdsourcing to enlist label information effectively and efficiently. Specifically, the adoption of crowdsourcing has been witnessed in a wide range of high-impact real-world applications, such as collaborative knowledge (e.g., data annotation, language translation), collective creativity (e.g., analogy mining, crowdfunding), and reverse Turing test (e.g., CAPTCHA-like systems), etc. Researchers and practitioners have mainly worked on the collaborative information aggregation problem where the data items are outsourced to a group of unskilled online workers for labeling. Despite the wide adoption of crowdsourcing services, several of its fundamental problems remain unsolved especially at the information and cognitive levels in terms of incentive design, aggregation methodology, and model learning. Specifically, the related computational challenges fall into the following categories of problems: (P1) Truth inference - How to infer the ground truth label of each item that has been distributed and labeled by multiple crowdsourcing labelers? What is the benefit of using truth inference for the downstream learning tasks? (P2) Heterogeneous learning - How to perform a unified model learning when the labeled data demonstrate multiple types of heterogeneity, such as oracle heterogeneity (more than one labels are collected from different labelers per item), task heterogeneity (multiple similar predictive tasks share commonality and are jointly learned), and view heterogeneity (items are characterized by different sources of features)? (P3) Crowd teaching - How to improve the labeling expertise of the crowdsourcing workers in the form of teaching and as a response, benefit the model learning eventually? The teaching mechanism should be personalized, effective, and interpretable. (P4) Recommendation - How to rank the user relevance over the unseen items if only a small number of user’s ratings (instead of the class labels in crowdsourcing) over items are given? How to properly handle the data sparsity, imbalanced sampling, and provide explanations to justify the recommendations? In this thesis, an in-depth study of these four problems is properly motivated, rigorously formulated, thoroughly evaluated, and broadly discussed. Specifically, one tensor augmentation and completion model is proposed to tackle the problem of Truth Inference; two unified optimization frameworks are proposed to solve the Heterogeneous Learning problem with crowdsourced labels that have various types of data heterogeneities - task heterogeneity and view heterogeneity; two Crowd Teaching mechanisms are proposed to guide crowd workers toward the personalized teaching as well as the interpretable concept learning. In the end, we tackle the Recommendation problem by proposing one unified ranking model that combines positive-unlabeled learning with generative adversarial networks to mitigate the data sparsity in collaborative filtering and also one systematical study regarding the explainability of the contextualized recommender systems. Optimizing the wisdom of the crowd is a promising direction with lots of potentials. Towards the end of the thesis, we have also presented a few promising future research directions such as complementary recommendations, fairness recommendations, and federated learning.
Graduation Semester
2022-05
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/115353
Copyright and License Information
Copyright 2022 Yao Zhou

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois