Adversarial graph contrastive learning with information regularization

Feng, Shengyu

Permalink

https://hdl.handle.net/2142/115773 Copy

Description

Title

Adversarial graph contrastive learning with information regularization

Author(s)

Feng, Shengyu

Issue Date

2022-04-25

Director of Research (if dissertation) or Advisor (if thesis)

Tong, Hanghang

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• graph representation learning
• contrastive learning
• adversarial training
• mutual information

Abstract

Contrastive learning is an effective unsupervised method in graph representation learning, and the key component of contrastive learning lies in the construction of positive and negative samples. Previous methods usually utilize the proximity of nodes in the graph as the principle to decide the positive and negative samples. Recently, the data augmentation based contrastive learning method has advanced to show great power in the visual domain. This method forces the encoder to maximize the mutual information between the positive pairs generated through the data augmentation of the same instance. It achieves comparable performance to the supervised learning on various downstream tasks. Some works also extended this method from images to graphs. However, most prior works are directly adapted from the models designed for images. Unlike the data augmentation on images, the data augmentation on graphs is far less intuitive and much harder to provide high-quality contrastive samples, which are the key to the performance of contrastive learning models. Although some works also try to design an adaptive data augmentation strategy based on human knowledge, they can only take effect on some specific datasets and fail to achieve a consistent improvement. This leaves much space for improvement over the existing graph contrastive learning frameworks. In this thesis, by introducing an adversarial graph view for data augmentation, we propose a simple but effective method, Adversarial Graph Contrastive Learning (ARIEL), to extract informative contrastive samples within a reasonable constraint. We make use of the Projected Gradient Descent attack to generate a new augmentation view by adding perturbations on the original graph, which maximizes the contrastive loss against a standard augmentation view, subject to the constraints on perturbations. Due to the instability brought by the adversarial view, we also develop a technique called information regularization to penalize the high similarity granted to a less similar pair. By using the subgraph sampling technique, we successfully scale ARIEL to large graphs with affordable computational cost. ARIEL consistently outperforms the current graph contrastive learning methods in the node classification and graph classification tasks over various real-world datasets. We further demonstrate that ARIEL could behave better on the attacked graphs than previous graph contrastive learning algorithms, which indicates ARIEL is more robust to adversarial attacks.

Graduation Semester

2022-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Shengyu Feng

Owning Collections