Distributional graph: Connecting language towards a representation of knowledge and meaning

Mao, Shufan

Permalink

https://hdl.handle.net/2142/122021 Copy

Description

Title

Distributional graph: Connecting language towards a representation of knowledge and meaning

Author(s)

Mao, Shufan

Issue Date

2023-11-30

Director of Research (if dissertation) or Advisor (if thesis)

Willits, Jon A

Doctoral Committee Chair(s)

Willits, Jon A

Committee Member(s)

• Fisher, Cynthia L
• Federmeier, Kara D
• Montag, Jessica L
• Dell, Gary S

Department of Study

Psychology

Discipline

Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Semantic memory
• Distributional semantics
• Knowledge representation
• Language comprehension
• Semantic Network

Abstract

What is the nature of the relationship between Knowledge and Language How does knowledge representation interact with language use? The critical role of world knowledge (lexical semantic) in language processing has long been noticed by psycholinguists, but relatively less attention has been paid to incorporating rich linguistic information in modeling knowledge and semantic representations. Current distributional models do form semantic representations from linguistic input. Despite their success in representing relatively simple semantic relations, these models are less effective in extracting rich linguistic structures from corpus, resulting in limited capability in representing complex lexical dependencies, achieving challenging semantic tasks, and accounting for relatively involved semantic behaviors. In this dissertation, I develop a novel type of distributional model – Distributional Graph – that transforms raw linguistic input into graphical forms and connects the graphlets to build a semantic network. The model encodes distributional patterns of linguistic units by a graphical topology, so that linguistically expressed concepts can be evaluated by network metrics. In particular, I adopt a spreading activation algorithm that gives rise to a graded measure of semantic relatedness on the network. I show, with two groups of studies, that distributional graphs equipped with spreading activation may better represent complex lexical relationships, compared to existing distributional models. In particular, I show that (i) The graphical encoding of co-occurrence leads to effective representation of indirect semantic relations which facilitates generalizing word-word lexical dependencies, and (ii) The explicit encoding of constituent structures in Distributional Graph leads to effective representation of multi-way lexical dependencies and success in compositional generalization tasks. The modeling works are conducted on artificially generated corpora with controlled distributional constraints, leading to a clear mechanistic account for the formal capabilities of the model. Meanwhile, the modeling results have profound implications on theory of language cognition and knowledge development in human. Considerable amount of behavioral studies are needed to validate Distributional Graph as a model for human semantic memory, and the experimental works require scaling up the Distributional Graph approach with naturalistic linguistic input. For the long-term vision, by integrating multi-modal inputs and finer grammatical information, the more full-fledged distributional graphs may give rise to generation of novel concepts that are both grammatical and meaningful. Importantly, as distributional graphs are based on explainable computational mechanisms, such advance may contribute to more interpretable AIs, and further the understanding of knowledge development and innovation in humanity.

Graduation Semester

2023-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Shufan Mao

Owning Collections