FreeBSD responding with wrong receiving interface IP

[Joe Holden]

On 10/03/2015 13:16, George Neville-Neil wrote:
> On 10 Mar 2015, at 11:26, Paul S. wrote:
>
>> Hi,
>>
>> I've been deploying FreeBSD as customer edge routers for customers
>> with sites that do not require high throughput (>1g/s).
>>
>> Each site has two ISPs (Mostly Telstra + Verizon/Optus), and take full
>> routes via OpenBGPd and BIRD. I use next-hop self on all received routes.
>>
>> The FreeBSD boxes have static routes delegating the announced IP
>> blocks to a L3 switch down the road. i.e: route add -net 10.100.1.0/24
>> 10.0.0.1, and then that /24 is originated via BGP to both upstreams.
>>
>> Things in general work fine, but I've been receiving reports of 'weird
>> traceroute results' from my customers.
>>
>> Examples of this would be,
>>
>> 1 some.random.isp (...) (...)
>> 2  gigabitethernet3-3.exi1.melbourne.telstra.net (203.50.77.49) 0.309
>> ms  0.284 ms  0.227 ms
>> 3  bundle-ether3-100.exi-core10.melbourne.telstra.net (203.50.80.1)
>> 1.966 ms  1.675 ms  1.852 ms
>> 4  bundle-ether12.chw-core10.sydney.telstra.net (203.50.11.124) 16.707
>> ms  15.917 ms  16.360 ms
>> 5  customer-gw.syd.ALTER.net (...) (...)
>>
>> This traceroute seems to claim that the packet was received over the
>> Verizon gateway, which in reality it was not -- it was received
>> directly over the Telstra interface, but my outbound AS-PATH towards
>> some.random.isp uses Verizon.
>>
>> So FreeBSD replies back with the Verizon address. Another person
>> having the same issue (mostly, but on OpenBSD) can be found at
>> http://openbsd.7691.n7.nabble.com/BGP-responding-with-wrong-IP-address-td90264.html
>>
>>
>> I would love to know if there's a way to fix this, or if I've missed
>> something, or if there's something wrong in the way I set it up.
>>
>> Thank you for taking the time to read.
>
> I wonder if we could see some routing tables?  That might help.
>
> Best,
> George

sysctl net.inet.icmp.reply_from_interface=1 will probably do what you 
expect.