On the beneficial impact of strong correlations for anomaly detection

Abstract

It is now widely accepted that packet network traffic exhibits long-range dependence (LRD), and this has been shown to be harmful to network performance. LRD also reduces the effectiveness of estimators of traffic parameters. For instance, it is much harder to estimate the mean of a LRD process than that of a process with only short-term correlations. One might intuitively expect that LRD would be detrimental to most networking tasks. One important network task is anomaly detection. Anomalies often correspond to problems, for instance, denial-of-service attacks or outages, and so rapid detection is important for maintaining a reliable network. In this article we demonstrate that, counter to the above intuition, LRD is actually beneficial to the detection of anomalies, as in fact are other forms of strong correlations in the observed process. We provide both theoretical proofs and simulation examples to show that LRD in traffic measurements actually improves the probability of detection of anomalies in that traffic.