Active heterogeneous graph neural networks with per-step meta-q-learning
Zhang, Yuheng
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/115599
Description
Title
Active heterogeneous graph neural networks with per-step meta-q-learning
Author(s)
Zhang, Yuheng
Issue Date
2022-04-26
Director of Research (if dissertation) or Advisor (if thesis)
Tong, Hanghang
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Active learning
Meta-Reinforcement learning
Heterogeneous graph neural network
Abstract
Recent years have witnessed the superior performance of heterogeneous graph neural networks (HGNNs) in dealing with heterogeneous information networks (HINs). Nonetheless, the success of HGNNs often depends on the availability of sufficient labeled training data, which can be very expensive to obtain in real scenarios. Active learning provides an effective solution to tackle the data scarcity challenge. Through actively acquiring the most informative samples, the performance of machine learning models could be greatly boosted with limited annotation cost. For the vast majority of the existing work regarding active learning on graphs, they mainly focus on homogeneous graphs, and thus fall in short or even become inapplicable on HINs. In this thesis, we study the active learning problem with HGNNs and propose a novel meta-reinforced active learning framework MetRA. We formulate the active learning process as a Markov Decision Process (MDP) and employ deep Q-learning to learn the labeling policy. Previous reinforced active learning algorithms train the policy network on labeled source graphs and directly transfer the policy to the target graph without any adaptation. To better exploit the information from the target graph in the adaptation phase, we propose a novel policy transfer algorithm based on meta-Q-learning termed per-step MQL. Specifically, we measure the similarity between the transitions from the meta-training replay buffer and the target graph state in each time step. The source transitions with high similarity to the target graph will be recycled to adapt our policy using off-policy updates. It is noteworthy to mention that our per-step MQL algorithm could be generalized to other reinforced active learning frameworks. Empirical evaluations on both HINs and homogeneous graphs demonstrate the effectiveness and efficiency of our proposed framework. The improvement over the best baseline is up to 7% in Micro-F1.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.