Skip to main navigation Skip to search Skip to main content

VulLibGen: Generating Names of Vulnerability-Affected Packages via a Large Language Model

  • Tianyu Chen
  • , Lin Li
  • , Liuchuan Zhu
  • , Zongyang Li
  • , Xueqing Liu
  • , Guangtai Liang
  • , Qianxiang Wang
  • , Tao Xie
    • Peking University
    • Huawei Technologies Co., Ltd.

    Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

    4 Scopus citations

    Abstract

    Security practitioners maintain vulnerability reports (e.g., GitHub Advisory) to help developers mitigate security risks. An important task for these databases is automatically extracting structured information mentioned in the report, e.g., the affected software packages, to accelerate the defense of the vulnerability ecosystem. However, it is challenging for existing work on affected package identification to achieve high precision. One reason is that all existing work focuses on relatively smaller models, thus they cannot harness the knowledge and semantic capabilities of large language models. To address this limitation, we propose VulLibGen, the first method to use LLM for affected package identification. In contrast to existing work, VulLibGen proposes the novel idea to directly generate the affected package. To improve the precision, VulLibGen employs supervised fine-tuning (SFT), retrieval augmented generation (RAG) and a local search algorithm. The local search algorithm is a novel post-processing algorithm we introduce for reducing the hallucination of the generated packages. Our evaluation results show that VulLibGen has an average precision of 0.806 for identifying vulnerable packages in the four most popular ecosystems in GitHub Advisory (Java, JS, Python, Go) while the best average precision in previous work is 0.721. Additionally, VulLibGen has high value to security practice: we submitted 60 <vulnerability, affected package> pairs to GitHub Advisory (covers four ecosystems) and 34 of them have been accepted and merged.

    Original languageEnglish
    Title of host publicationLong Papers
    EditorsLun-Wei Ku, Andre F. T. Martins, Vivek Srikumar
    Pages9767-9780
    Number of pages14
    ISBN (Electronic)9798891760943
    DOIs
    StatePublished - 2024
    Event62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024 - Bangkok, Thailand
    Duration: 11 Aug 2024 → 16 Aug 2024

    Publication series

    NameProceedings of the Annual Meeting of the Association for Computational Linguistics
    Volume1
    ISSN (Print)0736-587X

    Conference

    Conference62nd Annual Meeting of the Association for Computational Linguistics, ACL 2024
    Country/TerritoryThailand
    CityBangkok
    Period11/08/24 → 16/08/24

    Fingerprint

    Dive into the research topics of 'VulLibGen: Generating Names of Vulnerability-Affected Packages via a Large Language Model'. Together they form a unique fingerprint.

    Cite this