On event reproduction ratio in stateless and stateful replay of real-world traffic

Capturing and replaying network flows are important for testing network devices. Replayed traffic should reproduce effects similar to live traffic. This work presents methods to measure the event reproduction ratio, and studies the effectiveness of stateless and stateful traffic replayers based on the events triggered by packets and connections. We use two replayers, SocketReplay and Tcpreplay, and a networking device supporting security services. SocketReplay is a stateful replayer which keeps the state of a connection during replay, while Tcpreplay is a stateless replayer that ignores the connection state. Results indicate that SocketReplay replayed a smaller ratio of the captured traffic and triggered fewer blocking events in subsequent replay tests. Triggering blocking events denotes the replayed traffic cannot fit the onsite context. SocketReplay only replayed 38.74% of the captured TCP traffic, and resulted in an effectiveness of 99.97% (0.00%) in passing (blocking) event ratio. In contrast, Tcpreplay replayed 99.99% of the captured TCP traffic, and resulted in an effectiveness of 99.73% (75.64%) in passing (blocking) event ratio. The choice of a proper replayer and the corresponding replay configuration should depend on the contents of captured traffic and avoid to a significant drop of event reproduction ratio and the effectiveness of replayers.