Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks

Ping Wang, Khushbu Agarwal, Colby Ham, Sutanay Choudhury, Chandan K. Reddy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

55 Scopus citations

Abstract

Representation learning methods for heterogeneous networks produce a low-dimensional vector embedding (that is typically fixed for all tasks) for each node. Many of the existing methods focus on obtaining a static vector representation for a node in a way that is agnostic to the downstream application where it is being used. In practice, however, downstream tasks such as link prediction require specific contextual information that can be extracted from the subgraphs related to the nodes provided as input to the task. To tackle this challenge, we develop , a framework for bridging static representation learning methods using global information from the entire graph with localized attention driven mechanisms to learn contextual node representations. We first pre-train our model in a self-supervised manner by introducing higher-order semantic associations and masking nodes, and then fine-tune our model for a specific link prediction task. Instead of training node representations by aggregating information from all semantic neighbors connected via metapaths, we automatically learn the composition of different metapaths that characterize the context for a specific task without the need for any pre-defined metapaths. significantly outperforms both static and contextual embedding learning methods on several publicly available benchmark network datasets. We also demonstrate the interpretability, effectiveness of contextual learning, and the scalability of through extensive evaluation.

Original languageEnglish
Title of host publicationThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021
Pages2946-2957
Number of pages12
ISBN (Electronic)9781450383127
DOIs
StatePublished - 19 Apr 2021
Event2021 World Wide Web Conference, WWW 2021 - Ljubljana, Slovenia
Duration: 19 Apr 202123 Apr 2021

Publication series

NameThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021

Conference

Conference2021 World Wide Web Conference, WWW 2021
Country/TerritorySlovenia
CityLjubljana
Period19/04/2123/04/21

Keywords

  • Heterogeneous networks
  • Link prediction
  • Network embedding
  • Self-supervised learning
  • Semantic association

Fingerprint

Dive into the research topics of 'Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks'. Together they form a unique fingerprint.

Cite this