Generating descriptions for a structured knowledge base

Wang, Qingyun

Permalink

https://hdl.handle.net/2142/115750 Copy

Description

Title

Generating descriptions for a structured knowledge base

Author(s)

Wang, Qingyun

Issue Date

2022-04-26

Director of Research (if dissertation) or Advisor (if thesis)

Ji, Heng

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Knowledge Base
• Pre-trained Language Models
• Attention Mechanism
• Position Embeddings
• Table Position Self-Attention
• Two-step Fine-tuning Mechanism

Abstract

Generating natural language sentences to narrate structured knowledge automatically has been in great demand for human communication and knowledge propagation. We aim to automatically generate natural language descriptions about an input structured knowledge graph (KG). Graph-to-text generation has benefited from pre-trained language models (PLMs) in achieving better performance than structured graph encoders. However, they fail to fully utilize the structure information of the input graph. In this thesis, we first build our generation framework based on a pointer network which can copy facts from the input knowledge base (KB), and add two attention mechanisms: (i) slot-aware attention to capture the association between a slot type and its corresponding slot value; and (ii) a new table position self-attention to capture the inter-dependencies among related slots. We also create a new data set which includes 106,216 pairs of structured KBs and their corresponding natural language descriptions for two distinct entity types. We then aim to further improve the performance of the pre-trained language model by proposing a structured graph-to-text model with a two-step fine-tuning mechanism which first fine-tunes the model on Wikipedia before adapting to the graph-to-text generation. In addition to using the traditional token and position embeddings to encode the knowledge graph (KG), we propose a novel tree-level embedding method to capture the inter-dependency structures of the input graph. Experiments show that our approach significantly outperforms state-of-the-art methods.

Graduation Semester

2022-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Qingyun Wang

Owning Collections