A Heuristic Statistical Testing Based Approach for Encrypted Network Traffic Identification

Weina Niu, Zhongliu Zhuo, Xiaosong Zhang, Xiaojiang Du, Guowu Yang, Mohsen Guizani

Research output: Contribution to journalArticlepeer-review

29 Scopus citations

Abstract

In recent years, malware with strong concealment uses encrypted protocol to evade detection. Thus, encrypted traffic identification can help security analysts to be more effective in narrowing down those encrypted network traffic. Existing methods are protocol independent, such as statistical-based and machine-learning-based approaches. Statistical-based approaches, however, are confined to payload length and machine-learning-based approaches have a low recognition rate for encrypted traffic using undisclosed protocols. In this paper, we proposed a heuristic statistical testing (HST) approach that combines both statistics and machine learning and has been proved to alleviate their respective deficiencies. We manually selected four randomness tests to extract small payload features for machine learning to improve real-time performances. We also proposed a simple handshake skipping method called HST-R to increase the classification accuracy. We compared our approach with other identification approaches on a testing dataset consisting of traffic that uses two known, two undisclosed, and one custom cryptographic protocols. Experimental results showed that HST-R performs better than other traditional coding-based, entropy-based, and ML-based approaches. We also showed that our handshake skipping method could generalize better for unknown cryptographic protocols. Finally, we also conducted experimental comparisons among different classification algorithms. The results showed that C4.5, with our method, has the highest identification accuracy for secure sockets layer and secure shell traffic.

Original languageEnglish
Article number8620362
Pages (from-to)3843-3853
Number of pages11
JournalIEEE Transactions on Vehicular Technology
Volume68
Issue number4
DOIs
StatePublished - Apr 2019

Keywords

  • Encrypted traffic identification
  • handshake skipping
  • machine learning
  • protocol-independent
  • statistical testing

Fingerprint

Dive into the research topics of 'A Heuristic Statistical Testing Based Approach for Encrypted Network Traffic Identification'. Together they form a unique fingerprint.

Cite this