vnet Jail on a non-dedicated network interface

[Kristof Provost]

On 14 Oct 2020, at 15:36, Arsenij Solovjev wrote:
> On Wed, 14 Oct 2020 at 14:42, Kristof Provost <kp at freebsd.org> wrote:
>
>> On 14 Oct 2020, at 14:18, Arsenij Solovjev wrote:
>>> Hi all!
>>> Does anybody know if it's possible to run a vnet jail on a
>>> non-dedicated
>>> interface? I have the Lucas book on jails. In it he says that for 
>>> vnet
>>> you
>>> need to pick a dedicated interface, remove all networking IP
>>> configuration
>>> and only bring it up. Afterwards you set up jib and whatnot.
>>>
>>> All works well if I use a dedicated secondary interface (let's call 
>>> it
>>> em1). If I use em0 however I cannot ping the jail.
>>>
>>> I would like to have a host with that has a single network interface
>>> which
>>> is used for both normal networking stuff as well as having the vnet
>>> jail
>>> run on it.
>>>
>>> Maybe I could create some sort of virtual interface and run vnet on
>>> it?
>>>
>>> Any ideas here? Thanks in advance!
>>>
>> Look at epair interfaces.
>>
>> You can put em0 and epair0a in a bridge together and add epair0b to 
>> the
>> vnet jail.
>> That gets the vnet jail connected to your LAN.
>>
>> Or you can skip the bridge, assign an IP to epair0a and route between
>> the jail and your LAN.
>>
>> Regards,
>> Kristof
>>
>
> Hi Kristof,
>
> Thanks for your reply!
>
> considering your first idea. I did this, the jail gets created 
> seemingly
> fine. However I cannot ping the ip of epair0b (this works when using a
> dedicated interface).
> Also I cannot reach my gateway from within the jail. This too works 
> when
> using a dedicated interface.
> Btw I have "sysctl security.jail.allow_raw_sockets=1".
> Here is my host ifconfig when putting em0 and epair0a in a bridge:
>
> em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 
> 0
>> mtu 1500
>>
> options=812099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
>
> ether 9a:4c:eb:b5:95:bf
>
> inet 172.18.20.145 netmask 0xffffff00 broadcast 172.18.20.255
>
> media: Ethernet autoselect (1000baseT <full-duplex>)
>
> status: active
>
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>
> jailether: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 
> mtu
>> 1500
>>
> options=81209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
>
> ether 56:39:b7:c5:2e:ec
>
> media: Ethernet autoselect (1000baseT <full-duplex>)
>
> status: active
>
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>>
> options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
>
> inet6 ::1 prefixlen 128
>
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
>
> inet 127.0.0.1 netmask 0xff000000
>
> inet 10.43.84.1 netmask 0xffffff00
>
> groups: lo
>
> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>
> em0bridge: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 
> mtu
>> 1500
>>
> ether 02:13:0b:48:53:00
>
> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
>
> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>
> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>
> member: e0a_sambaad flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>
>        ifmaxaddr 0 port 5 priority 128 path cost 2000
>
> member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>
>        ifmaxaddr 0 port 1 priority 128 path cost 20000
>
> groups: bridge
>
> nd6 options=1<PERFORMNUD>
>
> e0a_sambaad: 
> flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
>> metric 0 mtu 1500
>>
> options=8<VLAN_MTU>
>
> ether 02:a4:c4:b5:95:bf
>
> hwaddr 02:78:fd:34:e8:0a
>
> groups: epair
>
> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
>
> status: active
>
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>
>
>
> Here's the ifconfig from my within my jail:
>
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>>
> options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
>
> inet6 ::1 prefixlen 128
>
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
>
> inet 127.0.0.1 netmask 0xff000000
>
> groups: lo
>
> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>
> e0b_sambaad: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 
> 0
>> mtu 1500
>>
> options=8<VLAN_MTU>
>
> ether 0e:a4:c4:b5:95:bf
> hwaddr 02:78:fd:34:e8:0b
>
This is odd. Are you assigning a new MAC address to the epair interfaces 
somewhere? Both ends of the epair seem to have a new MAC address, and 
the same one at that.

Regards,
Kristof