The Power of Slicing in Internet Flow Measurement

Ramana Rao Estan Kompella Cristian
CS2005-0822
May 13, 2005

Flow measurement evolved into the primary method for measuring the composition of Internet traffic. Large ISPs and small networks use it to track dominant applications, dominant users, and traffic matrices. Cisco's NetFlow is a widely deployed flow measurement solution that uses a configurable static sampling rate to control processor and memory usage on the router and the amount of reporting traffic generated. Proposed enhancements to the basic sampled NetFlow solve various problems. For example, smart sampling reduces the overhead of reporting and storing the flow records generated by NetFlow by sampling them with probability proportional to their byte counts. Adaptive NetFlow limits memory and CPU consumption at the router by dynamically adapting the sampling rate used by NetFlow. In this paper we propose ``flow slices'', a flow measurement solution that can be deployed through a software update at routers and traffic analysis workstations. Flow slices borrows ideas from smart sampling and adaptive NetFlow, but it introduces significant new ideas too: a flow measurement algorithm related to sample and hold; new estimators for the number of active flows; basing smart sampling decisions on multiple factors; separating sampling rate adaptation from measurement bins; controlling the three resource bottlenecks at the router (CPU, memory, reporting bandwidth) using independent ``tuning knobs''. The resulting solution has smaller resource requirements than current proposals and it enables more accurate traffic analysis results. We provide theoretical analyses of the variances of the estimators based on the flow slices and experimental comparisons with other flow measurement solutions.


How to view this document


The authors of these documents have submitted their reports to this technical report series for the purpose of non-commercial dissemination of scientific work. The reports are copyrighted by the authors, and their existence in electronic format does not imply that the authors have relinquished any rights. You may copy a report for scholarly, non-commercial purposes, such as research or instruction, provided that you agree to respect the author's copyright. For information concerning the use of this document for other than research or instructional purposes, contact the authors. Other information concerning this technical report series can be obtained from the Computer Science and Engineering Department at the University of California at San Diego, techreports@cs.ucsd.edu.


[ Search ]


NCSTRL
This server operates at UCSD Computer Science and Engineering.
Send email to webmaster@cs.ucsd.edu