Automatic analysis and classification of obfuscated bot binaries

Ying-Dar Lin, Yi Ta Chiang*, Yu-Sung Wu, Yuan Cheng Lai

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Botnets is a serious threat to Internet security. Popular defense strategies such as traffic filtering and malware detection all require a good understanding of the constituent bot binaries for creating the corresponding filter rules or signatures. This means that an effective analysis and classification process for bot binaries is needed for dealing with the threat of botnets. Unfortunately, the rampant usage of binary obfuscation these days has made the analysis and classification rather difficult. A simple string pattern matching or disassembly of the binary no longer suffices as the exact instruction sequence can be easily altered by obfuscation. In this work, we propose a new framework for automatic analysis and classification of bot binaries. The framework analyzes a bot binary's runtime system call trace and uses the longest common subsequences between system call traces for the classification of bot binaries. The framework can effectively deal with obfuscated bot binaries. Experiment result shows that the framework can attain an overall 94% true positive rate and 93% true negative rate.

Original languageEnglish
Pages (from-to)477-486
Number of pages10
JournalInternational Journal of Network Security
Volume16
Issue number6
DOIs
StatePublished - 1 Jan 2014

Keywords

  • Longest common subsequence algorithm
  • Obfuscation
  • System call

Fingerprint

Dive into the research topics of 'Automatic analysis and classification of obfuscated bot binaries'. Together they form a unique fingerprint.

Cite this