Please help with PF-based redirector

[Alex Povolotsky]

Max Laier wrote:
> On Sunday 15 April 2007 20:11, Alex Povolotsky wrote:
>   
>> Hello!
>>
>> I'm trying to set up a box as round-robin TCP proxy. Of course, I'm
>> trying to do everything on kernel-level.
>>
>> This simple setup
>>
>> rdr on sk0 proto tcp from any to any port = smtp -> <outbound> port 25
>> round-robin
>>
>> should work. At least, I thought so.
>>
>> However, attempt to connect to port 25 yielded unexpected result. pfctl
>> -s state shows
>>
>> self tcp 89.108.94.212:25 <- 89.108.94.91:25 <-
>> 89.108.94.211:56975       CLOSED:SYN_SENT
>>     
>
> Your test hosts seem to be on the same subnet.  This does not work as you 
> seems to think.  In the same broadcast domain it is not possible for the 
> pf box to forward the packet on behalf of the sending host (otherwise it 
> would confuse the recipient or the switch).  Instead it emits icmp 
> redirects which are ignored in a normal setup.
>
> You have to separate the two networks in order for redirect to work the 
> way you want it to.
>   

Sorry, things are not THAT simple.

I've tried the setup:

unknown-1717# ifconfig
rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        options=8<VLAN_MTU>
        inet 10.180.210.2 netmask 0xffffff00 broadcast 10.180.210.255
        ether 00:0e:2e:98:7e:55
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
sk0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        options=b<RXCSUM,TXCSUM,VLAN_MTU>
        inet 89.108.94.91 netmask 0xfffff000 broadcast 89.108.95.255
        inet 10.180.220.1 netmask 0xffffff00 broadcast 10.180.220.255
        ether 00:18:f3:5c:de:6d
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
plip0: flags=108810<POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT> mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
        inet 127.0.0.1 netmask 0xff000000
pfsync0: flags=0<> mtu 1348
        syncpeer: 224.0.0.240 maxupd: 128
carp0: flags=49<UP,LOOPBACK,RUNNING> mtu 1500
        inet 89.108.94.92 netmask 0xfffff000
        carp: MASTER vhid 21 advbase 1 advskew 0

unknown-1717# pfctl -s nat
No ALTQ support in kernel
ALTQ related functions disabled
nat on sk0 inet from 10.180.210.0/24 to any -> (sk0) round-robin
rdr on rl0 proto tcp from any to any port = smtp -> <outbound> port 25 
round-robin

seems reasonable. yes?

FIRST connect works ok

Than - no success at all for some time. Than - works again

unknown-1717# pfctl -s state
No ALTQ support in kernel
ALTQ related functions disabled
self tcp 89.108.65.126:25 <- 10.180.210.2:25 <- 10.180.210.1:62736       
CLOSED:SYN_SENT
self tcp 89.108.65.126:25 <- 10.180.210.2:25 <- 10.180.210.1:58177       
FIN_WAIT_2:FIN_WAIT_2
self tcp 89.108.94.212:25 <- 10.180.210.2:25 <- 10.180.210.1:57950       
FIN_WAIT_2:FIN_WAIT_2
self tcp 89.108.94.212:25 <- 10.180.210.2:25 <- 10.180.210.1:58727       
CLOSED:SYN_SENT
self tcp 89.108.65.124:25 <- 10.180.210.2:25 <- 10.180.210.1:63480       
CLOSED:SYN_SENT
self tcp 10.180.210.1:63480 -> 89.108.94.91:54490 -> 
89.108.65.124:25       SYN_SENT:CLOSED
self tcp 10.180.210.1:62736 -> 10.180.220.1:52675 -> 
89.108.65.126:25       SYN_SENT:CLOSED
self tcp 10.180.210.1:58177 -> 89.108.94.91:51550 -> 
89.108.65.126:25       FIN_WAIT_2:FIN_WAIT_2
self tcp 10.180.210.1:58727 -> 10.180.220.1:50704 -> 
89.108.94.212:25       SYN_SENT:CLOSED
self tcp 10.180.210.1:57950 -> 89.108.94.91:65245 -> 
89.108.94.212:25       FIN_WAIT_2:FIN_WAIT_2

You can see that some connections works, and some fails. 110% problem is 
NOT on real SMTP servers' side.

Alex.