BSD as routing device for 2 ISPs

[Mario Lobo]

Hi;

I have a FreeBSD 8 STABLE doing just that!

On Sat, 14 Jun 2014 17:25:41 +0200
"no at spam@mgedv.net" <nospam at mgedv.net> wrote:

> hi,
> 
> although i had a look on pfsense, openbgpd, setfib(1) ideas and such,
> googlin' around and discussing with nw-admins for hours, i still don't
> really see a clear path for setting up a proper solution which is not
> sort of "tinkering" but still based on free OS's.

Not possible! You will have to tinker it, starting by recompiling the
kernel with  options    ROUTETABLES=whatever.

> 
> situation:
> we have 2 independent ISPs, each running it's own router/ext-ip-block.
> e.g. ISP A: IP 1.1.1.10-1.1.1.20, ISP B: IP 2.2.2.50-2.2.2.60.
> 

Almost exactly  my situation.


> goal 1: inside->outside:
> - NAT and spread traffic load-based across ISPs to use both wires

I've done it like this:

nat on $ext_if1 from ! ($ext_if1) to any -> ($ext_if1) port 1024:65535
nat on $ext_if2 from ! ($ext_if2) to any -> ($ext_if2) port 1024:65535

[snip..]

pass in on $int_if route-to { ($ext_if1 $ext_gw1), ($ext_if2
$ext_gw2) } round-robin sticky-address inet proto { tcp, udp } from any
to ! $int_if:network.

This balances the traffic beautifully between both ISPs

> - switch to "living" ISP in case the other goes down
> (loosing active connections is ok and will of course happen)


Ahh ! heavy tinkering here. 

I've developed a daemon that keeps testing both links for
connectivity, that acts together with a series of scripts, that
re-writes/reapplies the whole pf.conf, directing everything to the link
that is working.

I also have two squids running, one for each ISP. Traffic is
round-robin redirected from the inside to them.


> goal 2: outside->inside:
> - NAT different external IPs to the SAME service inside
> (eg. smtp: NAT 1.1.1.11:25 and 2.2.2.51:25 to 192.168.10.10:25)
> - allow connecting to the same service via different routes
> simultaneously eg: ssh from 8.8.8.8->1.1.1.12:22
> while ssh from 9.9.9.9->2.2.2.12:22,
> both end up NAT'd at 192.168.10.20:22.

That's even simpler. Redirect the traffic on each ext_if to the ssh
daemon.

rdr pass on $ext_if1 inet proto tcp to port 22 -> 192.168.10.20 port
22

rdr pass on $ext_if2 inet proto tcp to port 22 -> 192.168.10.20 port
22

You will know your external IPs so you choose the link.

> 
> goal 3: firewalling:
> either this box is the firewall, or any other idea welcome.
> (currently, there's a separate hw-firewall running which does NAT,
> too)

In my case, it is THE firewall.

> oh, and the box will be run as virtual machine's guest OS.
> 

That shouldn't be a problem but test, test and test. Depending on the
hypervisor, results could be different.

In my case, it is a physical machine.

I hope this helps.

-- 
Mario Lobo
http://www.mallavoodoo.com.br
FreeBSD since 2.2.8 [not Pro-Audio.... YET!!] (99% winblows FREE)
 
"UNIX was not designed to stop you from doing stupid things, 
because that would also stop you from doing clever things."