How do I balance bandwidth over several virtual NICs?

[Elof Ofel]

Hi Adrian!

Now this sounds promising! All my sensors use the ixgbe driver.
However, my skills in programming/compiling isn't vast. I know how to patch and use poudriere. That's about it.

I must admit I don't really understand what you mean with "patch it to use a symmetric RSS key", but it sounds like the functionality I'm looking for is not yet there in the driver.

If we assume that someone in the future write and submit the above into the ixgbe driver, could I be so bold as to ask you for a commandline/configuration example (a brief guide) of how one would setup netmap and how to configure it to use the RX-queues?

That way I can start playing around with netmap and learning it while I wait for the ixgbe driver to be updated... I've got two professional programmer colleagues who've dealt extensively with e.g. the libnids and pfring source code, so if I get a grasp of how to setup netmap, and I find it interesting, it is likely that they can dive into and fix the ixgbe driver and improve it as per above. So please, can you help me with a "netmap guide"?

When I try to find documentation or examples of how to setup netmap I find none. Not even the netmap-enabled pcaplib contain any information as how to use it. I'm no programmer, so showing me different C structs for delivering data is of no use. :-/ 


I would very much like to improve the ixgbe driver and give back to the FreeBSD community rather than scrap FreeBSD and move to Linux and PF-RING.

/Elof

> Date: Mon, 22 Sep 2014 12:46:01 -0700
> Subject: Re: How do I balance bandwidth over several virtual NICs?
> From: adrian at freebsd.org
> To: elofu17 at hotmail.com
> CC: nike_d at cytexbg.com; freebsd-net at freebsd.org
> 
> Hi,
> 
> Yes.
> 
> * grab an ixgbe NIC and the -HEAD driver; (or cxgbe - I haven't gone
> and written RSS programming code for that just yet);
> * patch it to use a symmetric RSS key;
> * configure up N queues;
> * run an instance of snort on each TX/RX ring from the NIC.
> 
> The last step requires that you have snort use netmap rather than just
> straight bpf - or maybe somehow there's a way to glue bpf into a
> single netmap ring.
> 
> I haven't wrapped all of this up and thrown it into FreeBSD-HEAD yet,
> but i know that a symmetric RSS key works fine on 82599 hardware with
> a fixed driver.
> 
> 
> -a
> 
> 
> On 22 September 2014 12:06, Elof Ofel <elofu17 at hotmail.com> wrote:
> > Hi Nikolay.
> >
> > Unfortunetly no, that's not a solution.
> > mon0 could in theory be a bridge0 with four 10 GE interfaces = 40 Gbps theoretical input that need to be distributed over multiple virtual NICs. Also, I have no control of the mirrored traffic, so it would be hard for me to build and maintain bpf filters that tries to roughly balance the bandwidth load.
> >
> > Any other suggestions?
> >
> > /Elof
> >
> >> Date: Mon, 22 Sep 2014 18:45:28 +0200
> >> Subject: Re: How do I balance bandwidth over several virtual NICs?
> >> From: nike_d at cytexbg.com
> >> To: elofu17 at hotmail.com
> >> CC: freebsd-net at freebsd.org
> >>
> >> On Mon, Sep 22, 2014 at 5:12 PM, Elof Ofel <elofu17 at hotmail.com> wrote:
> >> > I have a single NIC, mon0, that constantly receive 800 Mbps of mirrored traffic.
> >> > I want to split these 800 Mbps into smaller chunks and feed them to a couple of virtual interfaces.
> >> > Each virtual interface can then have instance of 'snort' inspecting its traffic.
> >> >
> >> > Say approximately 200 Mbps per interface = four interfaces.
> >> > That way, each of the four snort processes only get 200 Mbps of data to inspect instead of having *one* single snort process (single-threaded) trying to cope with 800 Mbps.
> >> >
> >> > (the problem I'm trying to solve is utilizing all cpu's. Currently one cpu runs snort at 100% while all the other cpu's idle.)
> >> >
> >> >
> >> > The important thing though is that all packets in the connection need to be diverted to the same virtual NIC. You can't send the SYN to NIC0 and the SYN-ACK to NIC1, 'cause then neither snort-process-0 nor snort-process-1 see the other side of the connection.
> >> > The loadbalancing must be based on a hash built from at least the mac-addresses+IP-addresses.
> >> >
> >> >
> >> > So, what I think I'm looking for is a way to configure a lagg0 interface in loadbalance mode, that take all the incoming traffic on mon0 and distribute it over four virtual member NICs. (these four NICs would then probably be configured to run in monitor mode.)
> >> >
> >> >
> >> > Do FreeBSD support what I'm looking for? How do I do it? Where should I look?
> >> >
> >> > /Elof
> >> >
> >> > _______________________________________________
> >> > freebsd-net at freebsd.org mailing list
> >> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> >> > To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> >>
> >> Since this is below one Gig, would running separate snort processes on
> >> mon0 and using a BPF filter to split traffic work?
> >>
> >> --Nikolay
> >
> > _______________________________________________
> > freebsd-net at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-net
> > To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"