Multi-Faceted Knowledge-Driven Pre-Training for Product Representation Learning

Denghui Zhang, Yanchi Liu, Zixuan Yuan, Yanjie Fu, Haifeng Chen, Hui Xiong

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

As a key component of e-commerce computing, product representation learning (PRL) provides benefits for a variety of applications, including product matching, search, and categorization. The existing PRL approaches have poor language understanding ability due to their inability to capture contextualized semantics. In addition, the learned representations by existing methods are not easily transferable to new products. Inspired by the recent advance of pre-trained language models (PLMs), we make the attempt to adapt PLMs for PRL to mitigate the above issues. In this article, we develop KINDLE, a Knowledge-drIven pre-trainiNg framework for proDuct representation LEarning, which can preserve the contextual semantics and multi-faceted product knowledge robustly and flexibly. Specifically, we first extend traditional one-stage pre-training to a two-stage pre-training framework, and exploit a deliberate knowledge encoder to ensure a smooth knowledge fusion into PLM. In addition, we propose a multi-objective heterogeneous embedding method to represent thousands of knowledge elements. This helps KINDLE calibrate knowledge noise and sparsity automatically by replacing isolated classes as training targets in knowledge acquisition tasks. Furthermore, an input-aware gating network is proposed to select the most relevant knowledge for different downstream tasks. Finally, extensive experiments have demonstrated the advantages of KINDLE over the state-of-the-art baselines across three downstream tasks.

Original languageEnglish
Pages (from-to)7239-7250
Number of pages12
JournalIEEE Transactions on Knowledge and Data Engineering
Volume35
Issue number7
DOIs
StatePublished - 1 Jul 2023

Keywords

  • Product representation learning
  • pre-trained language models
  • product classification
  • product matching
  • product search

Fingerprint

Dive into the research topics of 'Multi-Faceted Knowledge-Driven Pre-Training for Product Representation Learning'. Together they form a unique fingerprint.

Cite this