Relation extraction: exploring syntax parsing and constructing it as attention-like structure
Huang, Rui
Loading…
Permalink
https://hdl.handle.net/2142/113346
Description
Title
Relation extraction: exploring syntax parsing and constructing it as attention-like structure
Author(s)
Huang, Rui
Issue Date
2021-07-21
Director of Research (if dissertation) or Advisor (if thesis)
Sun, Ruoyu
Department of Study
Industrial&Enterprise Sys Eng
Discipline
Industrial Engineering
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Natural Language Processing
Syntax Parsing
Relation Extraction
BERT
Abstract
Relation extraction has attracted scientists’ attention since early 21st centuries and it has been one of the common natural language processing (NLP) tasks. It is so important since it could extract semantic relationships from corpus. There are several subtasks in relation extraction area, including joint entity and relation recognition, dialog relation extraction and few-shot relation classification. Some literatures have conducted experiments on the combination of syntax parsing, attention mechanism and transformers and they had obtained outstanding improvement in their tasks. Motivated by application of syntax parsing and recent state of arts of syntax-aware BERT [1] related models, we construct a syntax mechanism to convert syntax tree structure to matrixes and apply on the finetuning process of SyntaxBERT [2] and syntax-aware -local-attention attention BERT (SLA) [3] to strengthen their ability to learning entity relations. They would be finetuned on TACRED [4] dataset. They are compared with the finetuning of SpanBERT [5] and SpanBERT+RECENT [6], which has obtained the best result in TACRED. The result of the experiments inspires some interesting thoughts on syntax parsing, attention mechanism and BERT.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
