Bridging crowd and machine intelligence to inspire causal reasoning

Yen, Chi-Hsien

Permalink

https://hdl.handle.net/2142/115588 Copy

Description

Title

Bridging crowd and machine intelligence to inspire causal reasoning

Author(s)

Yen, Chi-Hsien

Issue Date

2022-04-20

Director of Research (if dissertation) or Advisor (if thesis)

Huang, Yun

Doctoral Committee Chair(s)

• Huang, Yun
• Bailey, Brian P.

Committee Member(s)

• Karahalios, Karrie G.
• Wang, Hao-Chuan

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Causal Reasoning
• Causal Inference
• Crowd Intelligence
• Collaborative Reasoning
• Intelligent Systems
• Crowd-Informed Reasoning Systems
• Causal Diagrams and Narratives

Abstract

Causal reasoning, the process of identifying cause-effect relationships, is naturally performed by everyone in everyday life to explain phenomena, predict the future, and make decisions. With the rise of open data and visualization tools, people with varying data skills and domain knowledge now have the opportunity to infer causal relationships from data. However, causal reasoning remains challenging due to many factors, such as people's limited domain knowledge about the data and the difficulty of coming up with alternative hypotheses. This dissertation explores the effects of utilizing crowd and machine intelligence to address these challenges: a crowd could collectively provide relevant domain knowledge, diverse perspectives, and semantic explanations for data patterns, while machines could efficiently detect and present supporting or disconfirming data evidence to potential causal hypotheses. This dissertation consists of three parts, each utilizing user studies to investigate relevant topics on people's causal reasoning processes. First, it identified the factors that would impact the correctness of one's reasoning outcome and the common reasoning pitfalls people tend to make when inferring causal relationships from data visualizations. The results showed that an individual's blind spots, ineffective reasoning strategies, and inability to interpret data patterns are common pitfalls that could lead to inference errors. It also found that the participants' reasoning correctness decreased when a confounding variable impacted their reasoning target variable. Second, motivated by the first study's results, I explored how causal belief externalization and sharing could be enhanced by integrating causal diagrams and narratives in an interactive system. While diagrams are an effective graphical representation of causal relationships, narratives could explain detailed reasoning rationales and help users uncover their blind spots. I developed CausalIDEA, a system that allowed users to write narratives to rationalize their perceived causal relationships, visualize their causal models using directed diagrams, and review other peers' causal beliefs. The results showed that the participants' narratives could reveal their relevant, local, and diverse causal knowledge that provides examples or explanations for potential causal relationships. In addition, by reviewing others' causal beliefs, participants could identify blind spots and add new relationships to their causal models. Third, to further evaluate how the crowd's beliefs affect performing causal reasoning with data, I designed CrowdIDEA, a visual analytics system that bridges crowd and machine intelligence to assist causal analysis. CrowdIDEA allowed users to review causal beliefs collected from a crowd, inspect data visualizations and regressions results, and construct a causal diagram for a dataset. In the user study, I experimented with two designs for presenting the crowd's beliefs: the overview design, which provides an aggregated diagram based on the crowd's beliefs, and the focus design, which provides the most relevant crowd's beliefs regarding the user's reasoning focus. The results showed that the presentation design of the crowd's beliefs could lead to a conformity effect: the overview design led the participants to include more causal relationships in their diagrams, which made their causal diagrams more similar to the aggregated diagram. Further qualitative analysis identified the characteristics of the reasoning processes when people leverage the crowd's beliefs to develop hypotheses and shape their understanding of data. This dissertation provides empirical evidence and a deeper understanding of people's causal reasoning processes when crowd and machine intelligence are leveraged. As the crowd's beliefs have been shown to have various benefits through our studies, such as overcoming blind spots and facilitating hypotheses generation, I envision that crowds will be utilized more widely to enhance causal analysis. This dissertation provides a fundamental understanding and design implications for future crowd-informed causal reasoning systems.

Graduation Semester

2022-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Chi-Hsien Yen

Owning Collections