Attack lifecycle extraction and mapping from CTF writeups using an enhanced LLM approach

The MITRE ATT&CK framework defines an attack lifecycle that encapsulates the stages of an attacker’s actions, serving as an essential resource for recognizing and mitigating cyber threats. Accurately detecting MITRE ATT&CK techniques and attack lifecycles requires effective models trained on high-quality datasets. In this context, a high-quality dataset is one that provides representative coverage of attack techniques, balanced class distributions or effective mitigation of imbalance, and reliable ground-truth labels validated through manual annotation. Such properties are essential to ensure that the resulting models are both accurate and generalizable. Generating such datasets necessitates replaying complete attack lifecycles, which depend on well-defined lifecycle representations. Manually defining lifecycles is time-consuming and often infeasible due to the complexity of modern attacks. To address this, Capture The Flag (CTF) writeups, which offer detailed technical descriptions of attack steps, can serve as a resource for defining lifecycles. Automating the extraction and mapping of lifecycles from CTF writeups presents challenges such as data imbalance and the lack of existing approaches for identifying attack techniques, which are required for defining the lifecycle. This work aims to automate the extraction and mapping of attack lifecycles from CTF writeups by introducing Prefix Tuning, Augmentation, and Data Synthesis utilizing Large Language Models (PADS-LLM). This work is among the first to leverage CTF writeups for lifecycle extraction and introduces a unified framework combining prefix tuning, data augmentation, and multi-model synthesis. The framework efficiently classifies and aligns textual descriptions of attack steps into lifecycle representations, effectively addressing challenges related to limited data availability and domain-specific variations. Experimental results indicate that PADS-LLM achieves an accuracy of 50% across multiple reports and enhances the F1 score from 39.91% to 72.34%. These findings confirm that data augmentation and synthetic data generation are effective in mitigating class imbalance, while prefix tuning makes a significant contribution to domain adaptation. A notable observation is that integrating multiple language models further improves performance by capturing diverse variations of attack descriptions. PADS-LLM provides compelling evidence that large language models are capable of effectively automating the process of lifecycle generation from Capture The Flag (CTF) writeups. This automation not only reduces the need for manual intervention but also enhances the integrity and quality of the datasets used for machine learning-driven intrusion detection and adversary emulation.