TY - GEN
T1 - Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks
AU - Wang, Ping
AU - Agarwal, Khushbu
AU - Ham, Colby
AU - Choudhury, Sutanay
AU - Reddy, Chandan K.
N1 - Publisher Copyright:
© 2021 ACM.
PY - 2021/4/19
Y1 - 2021/4/19
N2 - Representation learning methods for heterogeneous networks produce a low-dimensional vector embedding (that is typically fixed for all tasks) for each node. Many of the existing methods focus on obtaining a static vector representation for a node in a way that is agnostic to the downstream application where it is being used. In practice, however, downstream tasks such as link prediction require specific contextual information that can be extracted from the subgraphs related to the nodes provided as input to the task. To tackle this challenge, we develop , a framework for bridging static representation learning methods using global information from the entire graph with localized attention driven mechanisms to learn contextual node representations. We first pre-train our model in a self-supervised manner by introducing higher-order semantic associations and masking nodes, and then fine-tune our model for a specific link prediction task. Instead of training node representations by aggregating information from all semantic neighbors connected via metapaths, we automatically learn the composition of different metapaths that characterize the context for a specific task without the need for any pre-defined metapaths. significantly outperforms both static and contextual embedding learning methods on several publicly available benchmark network datasets. We also demonstrate the interpretability, effectiveness of contextual learning, and the scalability of through extensive evaluation.
AB - Representation learning methods for heterogeneous networks produce a low-dimensional vector embedding (that is typically fixed for all tasks) for each node. Many of the existing methods focus on obtaining a static vector representation for a node in a way that is agnostic to the downstream application where it is being used. In practice, however, downstream tasks such as link prediction require specific contextual information that can be extracted from the subgraphs related to the nodes provided as input to the task. To tackle this challenge, we develop , a framework for bridging static representation learning methods using global information from the entire graph with localized attention driven mechanisms to learn contextual node representations. We first pre-train our model in a self-supervised manner by introducing higher-order semantic associations and masking nodes, and then fine-tune our model for a specific link prediction task. Instead of training node representations by aggregating information from all semantic neighbors connected via metapaths, we automatically learn the composition of different metapaths that characterize the context for a specific task without the need for any pre-defined metapaths. significantly outperforms both static and contextual embedding learning methods on several publicly available benchmark network datasets. We also demonstrate the interpretability, effectiveness of contextual learning, and the scalability of through extensive evaluation.
KW - Heterogeneous networks
KW - Link prediction
KW - Network embedding
KW - Self-supervised learning
KW - Semantic association
UR - http://www.scopus.com/inward/record.url?scp=85102798926&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85102798926&partnerID=8YFLogxK
U2 - 10.1145/3442381.3450060
DO - 10.1145/3442381.3450060
M3 - Conference contribution
AN - SCOPUS:85102798926
T3 - The Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021
SP - 2946
EP - 2957
BT - The Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021
T2 - 2021 World Wide Web Conference, WWW 2021
Y2 - 19 April 2021 through 23 April 2021
ER -