Few-Sample Named Entity Recognition for Security Vulnerability Reports by Fine-Tuning Pre-trained Language Models

Guanqun Yang, Shay Dineen, Zhipeng Lin, Xueqing Liu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

7 Scopus citations

Abstract

Public security vulnerability reports (e.g., CVE reports) play an important role in the maintenance of computer and network systems. Security companies and administrators rely on information from these reports to prioritize tasks on developing and deploying patches to their customers. Since these reports are unstructured texts, automatic information extraction (IE) can help scale up the processing by converting the unstructured reports to structured forms, e.g., software names and versions [8] and vulnerability types [38]. Existing works on automated IE for security vulnerability reports often rely on a large number of labeled training samples [8, 18, 48]. However, creating massive labeled training set is both expensive and time consuming. In this work, for the first time, we propose to investigate this problem where only a small number of labeled training samples are available. In particular, we investigate the performance of fine-tuning several state-of-the-art pre-trained language models on our small training dataset. The results show that with pre-trained language models and carefully tuned hyperparameters, we have reached or slightly outperformed the state-of-the-art system [8] on this task. Consistent with previous two-step process of first fine-tuning on main category and then transfer learning to others as in [7], if otherwise following our proposed approach, the number of required labeled samples substantially decrease in both stages: 90% reduction in fine-tuning from 5758 to 576, and 88.8% reduction in transfer learning with 64 labeled samples per category. Our experiments thus demonstrate the effectiveness of few-sample learning on NER for security vulnerability report. This result opens up multiple research opportunities for few-sample learning for security vulnerability reports, which is discussed in the paper. Our implementation for few-sample vulnerability entity tagger in security reports could be found at https://github.com/guanqun-yang/FewVulnerability.

Original languageEnglish
Title of host publicationDeployable Machine Learning for Security Defense - 2nd International Workshop, MLHat 2021, Proceedings
EditorsGang Wang, Arridhana Ciptadi, Ali Ahmadzadeh
Pages55-78
Number of pages24
DOIs
StatePublished - 2021
Event2nd International Workshop on Deployable Machine Learning for Security Defense, MLHat 2021, co-located with 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2021 - Virtual, Online
Duration: 15 Aug 202115 Aug 2021

Publication series

NameCommunications in Computer and Information Science
Volume1482 CCIS
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937

Conference

Conference2nd International Workshop on Deployable Machine Learning for Security Defense, MLHat 2021, co-located with 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2021
CityVirtual, Online
Period15/08/2115/08/21

Keywords

  • Few-sample named entity recognition
  • Public security reports
  • Software vulnerability identification

Fingerprint

Dive into the research topics of 'Few-Sample Named Entity Recognition for Security Vulnerability Reports by Fine-Tuning Pre-trained Language Models'. Together they form a unique fingerprint.

Cite this