Flow analysis tools

[Vincent Hoffman]

Daniel O'Connor wrote:
> Can anyone recommend some flow analysis tools?
> I am mostly interested in who (from inside my network) is downloading 
> how much (ie who is costing us money :)
>
> I have enabled netflow in mpd and I can capture it but I haven't really 
> found a suitable analysis tool yet.
>
> I tried nfsen and stager but I couldn't get them to break down based on 
> IP, just AS - not nearly fine grained enough for my needs.
>
>   
I'd have said nfdump/nfsen if you hadnt ruled them out.  I havent had
any problems getting IP level reports from it. I'f you are willing to
use the command line rather than the web interface its easy to cron a
daily report: for example, we only have netflow setup on the router to
do netflow for outgoing traffic from this location so for a report on
the usage of the top 20 source IPs (destination is interesting but
irrelevant for this exercise) Please note the -K option is just to
anonymize the IPs for this excercise.

[root at seaurchin ~/bin]# nfdump -M
/usr/local/var/nfsen/profiles-data/live/rsh1  -T  -K
sgjkouik67juhyt689076stegncitfds -R
2010/02/07/nfcapd.201002072355:2010/02/08/nfcapd.201002080945 -n 20 -s
srcip/bytes "src net 88.111.160.0/22"
Top 20 Src IP Addr ordered by bytes:
Date first seen          Duration Proto       Src IP Addr   
Flows(%)     Packets(%)       Bytes(%)         pps      bps   bpp
2010-02-07 23:54:33.133 35692.169 any      88.111.163.199     8566(
0.1)   53.2 M(31.9)   77.1 G(42.0)     1491   17.3 M  1447
2010-02-07 23:54:46.281 35718.377 any      88.111.163.156   329868(
2.1)   12.1 M( 7.3)   14.8 G( 8.1)      339    3.3 M  1219
2010-02-07 23:54:36.561 35722.425 any       88.111.162.41    52807(
0.3)    6.7 M( 4.0)    8.3 G( 4.5)      188    1.9 M  1233
2010-02-07 23:55:00.465 35699.941 any       88.111.163.36    28073(
0.2)    5.8 M( 3.5)    8.0 G( 4.3)      162    1.8 M  1372
2010-02-07 23:55:25.553 35639.753 any      88.111.163.158    90460(
0.6)    4.3 M( 2.6)    5.6 G( 3.1)      120    1.3 M  1315
2010-02-07 23:54:44.289 35725.773 any       88.111.162.65   166080(
1.0)    3.0 M( 1.8)    3.8 G( 2.1)       85   851674  1247
2010-02-07 23:55:09.025 35688.921 any       88.111.163.53    18458(
0.1)    2.6 M( 1.6)    3.7 G( 2.0)       73   820182  1396
2010-02-07 23:54:42.717 35718.217 any      88.111.160.160    10498(
0.1)    2.4 M( 1.4)    3.5 G( 1.9)       66   784449  1468
2010-02-07 23:55:02.245 35696.169 any       88.111.162.42   355185(
2.2)    3.5 M( 2.1)    2.9 G( 1.6)       96   640147   825
2010-02-07 23:54:51.069 35715.737 any      88.111.163.150   185657(
1.2)    2.8 M( 1.7)    2.8 G( 1.5)       79   631644   990
2010-02-07 23:55:09.053 35681.053 any       88.111.162.72     9766(
0.1)    1.3 M( 0.8)    1.8 G( 1.0)       36   410339  1399
2010-02-07 23:55:01.993 35698.869 any      88.111.162.113    53386(
0.3)    1.4 M( 0.9)    1.8 G( 1.0)       40   408840  1262
2010-02-07 23:53:15.833 35807.625 any       88.111.162.91   197756(
1.2)    2.5 M( 1.5)    1.7 G( 0.9)       70   385855   688
2010-02-07 23:55:14.321 35682.961 any      88.111.160.134     8925(
0.1)    1.2 M( 0.7)    1.7 G( 0.9)       34   370990  1352
2010-02-07 23:54:18.257 35744.017 any       88.111.163.69    82420(
0.5)    1.3 M( 0.8)    1.6 G( 0.9)       35   366048  1296
2010-02-07 23:54:02.605 35755.409 any      88.111.163.112   120445(
0.8)    1.6 M( 0.9)    1.6 G( 0.9)       43   355841  1024
2010-02-07 23:55:06.065 35686.533 any      88.111.162.132    20215(
0.1)    1.1 M( 0.7)    1.4 G( 0.8)       30   322267  1312
2010-02-07 23:53:40.965 35782.949 any      88.111.163.157   136187(
0.9)    1.5 M( 0.9)    1.4 G( 0.8)       40   315334   969
2010-02-07 23:55:09.173 35688.901 any      88.111.161.247    18823(
0.1)    1.0 M( 0.6)    1.4 G( 0.8)       28   313011  1357
2010-02-07 23:54:58.133 35701.225 any       88.111.162.25    25457(
0.2)    1.1 M( 0.6)    1.4 G( 0.8)       30   310943  1284

IP addresses anonymized
Summary: total flows: 15946440, total bytes: 183.7 G, total packets:
166.7 M, avg bps: 41.0 M, avg pps: 4654, avg bpp: 1101
Time window: 2010-02-07 23:53:15 - 2010-02-08 09:50:12
Total flows processed: 15946440, Blocks skipped: 0, Bytes read: 829226960
Sys: 4.534s flows/second: 3516363.2  Wall: 4.521s flows/second: 3526931.0


The same query is entirely doable in the nfsen web interface as well,
just a report with "stat topN", top: 20, stat: SRC IP Addresses order by
bytes, with a filter of "src net 88.111.160.0/22"
We also use a customer written alert plugin to detect pps based DOS/DDOS
attacks (if a single host exceeds 100K pps then tell us who it is and
who is sending the packets.)

pmacct isnt bad but I didnt like any of the interfaces I could find and
didnt want to write my own, plus storing it all in a mysql database
meant it had a much greater overhead than nfsen/dump.
ntop was fine for smaller traffic but went belly up pretty quickly at
higher traffic levels.
didnt try stager.
Flow tools looked ok if you want to roll your own, but no gain on nfdump
really just without nfsen.


Vince