OS choice for an edge router

[Bakul Shah]

> This is not the case.  Flood ping doesn't reach the limit in any
> way.  Have a look at the ping man page and flood ping description.

Ah yes, I was forgetting about the strict synchrony.

> Stock FreeBSD 6.2 or 7.0 can easily do 500kpps with good network
> cards and fastforwarding enabled.  On a dual-Opteron 2.6GHz with
> PCI-X Intel and Broadcom network cards I've done 800kpps in-out.

What is the throughput when fastforwarding is not used and
packets go to different destinations?  Note that typically
fastforwarding does not help much on a router since only one
route is cached.

> > Listen to what Louis Mamakos said!  Use FreeBSD primarily for
> > the control plane.  May be there are NICs where you can
> > offload some packet forwarding.... But that is a substantial
> > change to FreeBSD.  Or live with what FreeBSD can do on a
> > given box.
> 
> There are no NICs known that can do packet forwarding offload.
> And neither is there support in FreeBSD for that.  You're probably
> confusing this with checksum offloading or TSO (TCP segmentation
> offloading) which isn't an issue with packet forwarding at all.

Indeed.  That is why I said "that is a substantial change to
FreeBSD"!  But even offloading checksum can help as the CPU
has less to do.

> I'm running all my routing on FreeBSD since about 1998.  No
> problems and much more reliable than the countless Cisco IOS
> versions that have been deprecated since then.  On any more
> recent platform or new line card you have to run IOS T versions
> which is most of the time is much worse than running FreeBSD-current
> on a production machine.  It's probably cheaper to pay FreeBSD
> developers to fix any issues you find or run into than to pay
> Cisco for the pretty much mandatory service contract where any
> useful level starts at some 14% annually of the purchase price.
> And even then you have to pay for TAC cases and you are last in
> the queue relative to all others who pay more.

This is fine if he was building one or a few for his own
company's use but for selling routers to a third party you
have to productize the software and provide tighter bounds on
when you will fix critical bugs.  Also, what works for Cisco
won't work for a startup.  Even if you provide free service
they may not want to buy your product!

> Can't comment on VPN or IPSEC stuff.  Never used that to any
> significant extent.  However keep in mind that for the price
> of a single high powered Cisco or Juniper you can buy a very
> large number of also quite well powered FreeBSD powered routers.

Agreed!

> My recommendation for a optimal FreeBSD based router is as follows:
> CPU Core2 Duo or Athlon 64X2, more cores don't help in any way.  One
> core can take the interrupts and one can continue to serve userland.
> A quality mainboard from Tyan, Supermicro or Intel with PCI-Express.
> A number of (dual-port) Intel Gigabit PCI-E network cards.  Some two
> GB of RAM and a flash based ATA or SATA harddisk.  Good case, redundant
> power supplies, good fans and otherwise no movable parts.  Don't try
> RAID1 or stuff like that, causes more problems than it solves.  Go for
> a single flash disk that is replaceable without having to disassemble
> the entire case.  There are some 3.5" based flash disks on the market
> or buy a CF to ATA adapter for mounting into a 3.5" disk slot and use
> normal but fast CF cards.  That'll do it.

May be!