Enhancing diversity in generative commonsense reasoning for explaining relationships between concepts

Liu, Chenzhengyi

Permalink

https://hdl.handle.net/2142/120265 Copy

Description

Title

Enhancing diversity in generative commonsense reasoning for explaining relationships between concepts

Author(s)

Liu, Chenzhengyi

Issue Date

2023-04-16

Director of Research (if dissertation) or Advisor (if thesis)

Chang, Kevin Chen-Chuan

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Natural Language Processing
• Commonsense Reasoning
• Diverse Text Generation
• Large Language Models
• Mixture of Experts
• COLD Decoding

Abstract

The ability to reason based on common sense and knowledge of how things work is crucial for machines to navigate the world. However, large language models (LLMs) often lack explicit representations of relationships between concepts and events, making it challenging to interpret their reasoning processes. To overcome these challenges, in this paper, we propose DimonGen task, which aims to generate diverse sentences describing concept relationships in various everyday scenarios. To support this, we also create a new benchmark dataset for this task by extracting the existing ConceptNet and CommonGen dataset. To address the DimonGen task, we propose two complementary methods: MoREE, a two-stage method that utilizes external knowledge to generate diverse relationship sentences, and DC Decoding, a decoding framework that uses a global energy function to diversify the set of generations. Both methods are evaluated on the benchmark dataset and show significant improvements in the quality and diversity of generated sentences. The results suggest that these methods can generate diverse sentences that reflect relationships between concepts from multiple and varied perspectives. Our code and data for the DimmonGen task are available at https://github.com/liuchenzhengyi/DimonGen.

Graduation Semester

2023-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Chenzhengyi Liu

Owning Collections