cxgbe netmap promiscuous mode?
Vincenzo Maffione
v.maffione at gmail.com
Fri Mar 24 14:17:55 UTC 2017
Hi Joe,
From your description it's pretty clear that you hit the FreeBSD11 bug
that is now fixed in 11.0-stable (in addition to the upstream netmap
version on github).
The fix was committed in r312783.
Cheers,
Vincenzo
2017-03-24 14:50 GMT+01:00 Joe Jones <joe at stream-technologies.com>:
> Hi Vincenzo,
>
> we can get rid if the panic if we compile the kernel without netmap, then
> load an up to date netmap as a module. We can live with that for now.
>
> Some time last year before Free BSD 11 was branched from head, we compiled
> a checkout and it worked without problem on the same hardware setup. I may
> try and figure out what the regression is, I'm not familiar with the
> FreeBSD release process either though.
>
> Thanks,
> Joe Jones
>
> On 24/03/17 10:29, Vincenzo Maffione wrote:
>
>> Hi Joe,
>> There was a fix for a panic in emulated mode that was applied stable/11
>> branch, so I guess it also ended up into FreeBSD-11.0-STABLE.
>> I don't know whether the same fix ended up into in 11.0-RELEASE-p8 (I'm
>> not familiar with FreeBSD releasing process, sorry!).
>>
>> Or maybe this panic happens with netmap upstream?
>> If this is a new bug, it would be nice to have the kernel with the debug
>> symbols enabled, so that we can get more detailed information from the
>> stack trace.
>>
>> Cheers,
>> Vincenzo
>>
>> 2017-03-24 11:19 GMT+01:00 Joe Jones <joe at stream-technologies.com
>> <mailto:joe at stream-technologies.com>>:
>>
>>
>> Hi Vincenzo,
>>
>> I just tried with that sysctl set to 2, I get a similar looking
>> panic to before
>>
>> Fatal trap 12: page fault while in kernel mode
>> cpuid = 7; apic id = 0e
>> fault virtual address = 0x1
>> fault code = supervisor read data, page not present
>> instruction pointer = 0x20:0xffffffff806e38aa
>> stack pointer = 0x28:0xfffffe047ba18440
>> frame pointer = 0x28:0xfffffe047ba18490
>> code segment = base 0x0, limit 0xfffff, type 0x1b
>> = DPL 0, pres 1, long 1, def32 0, gran 1
>> processor eflags = interrupt enabled, resume, IOPL = 0
>> current process = 2205 (main)
>> trap number = 12
>> panic: page fault
>> cpuid = 7
>> KDB: stack backtrace:
>> #0 0xffffffff80b240f7 at kdb_backtrace+0x67
>> #1 0xffffffff80ad9462 at vpanic+0x182
>> #2 0xffffffff80ad92d3 at panic+0x43
>> #3 0xffffffff80fa1d51 at trap_fatal+0x351
>> #4 0xffffffff80fa1f43 at trap_pfault+0x1e3
>> #5 0xffffffff80fa14ec at trap+0x26c
>> #6 0xffffffff80f841c1 at calltrap+0x8
>> #7 0xffffffff806e5a80 at generic_netmap_txsync+0x330
>> #8 0xffffffff806e06f9 at netmap_ioctl+0x279
>> #9 0xffffffff8098624f at devfs_ioctl_f+0x13f
>> #10 0xffffffff80b41b34 at kern_ioctl+0x2d4
>> #11 0xffffffff80b417f1 at sys_ioctl+0x171
>> #12 0xffffffff80fa26ae at amd64_syscall+0x4ce
>> #13 0xffffffff80f844ab at Xfast_syscall+0xfb
>>
>> This is on 11.0-RELEASE-p8
>>
>> Thanks,
>> Joe Jones
>>
>> On 23/03/17 18:20, Vincenzo Maffione wrote:
>>
>> Hi,
>> You could try to use netmap in emulated mode (sysctl
>> dev.netmap.admode=2). If this works, at least you know that
>> the problem is in the cxgbe netmap support and not in the
>> netmap core itself.
>>
>> Cheers,
>> Vincenzo
>>
>> 2017-03-23 14:00 GMT+01:00 Joe Jones
>> <joe at stream-technologies.com
>> <mailto:joe at stream-technologies.com>
>> <mailto:joe at stream-technologies.com
>>
>> <mailto:joe at stream-technologies.com>>>:
>>
>> Hello,
>>
>> We have a T520-SO and have made a new install of 11.0, to
>> begin
>> with the box would panic every time we tried to switch the
>> card
>> into netmap mode. So we recompiled the kernel with netmap
>> removed,
>> then compiled the netmap kernel module from github, as
>> this in our
>> experience generally leads to a more stable netmap.
>>
>> we have
>>
>> uname -a
>> FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0:
>> Wed Mar
>> 22 16:52:35 UTC 2017 joe at goose2:/usr/obj/usr/src/sys/GENERIC
>> amd64
>>
>> and the following in /boot/loader.conf
>>
>> t4fw_cfg_load="YES"
>> t5fw_cfg_load="YES"
>> if_cxgbe_load="YES"
>> hw.cxgbe.fl_pktshift=0
>> hw.cxgbe.toecaps_allowed=0
>> hw.cxgbe.nnmtxq10g=8
>> hw.cxgbe.nnmrxq10g=8
>> hw.cxgbe.num_vis=2
>>
>> Before I run our application I run
>>
>> ifconfig cxl1 promisc -vlanhwtag up
>>
>> Now our application can now start without panicking the
>> kernel.
>> Here is where it gets interesting, our application is able to
>> announce it's self via ARP, I can see the ethernet switch
>> learning
>> which port it's on, and other hosts adding it to their ARP
>> tables.
>> When I try an ICMP ping it goes missing. After watching the TX
>> packet graph for the connected port on the switch while
>> starting
>> and stopping a flood ping to the application, I'm sure the
>> packets
>> are getting sent to the card, however I don't see them in the
>> netmap ring. If I kill our application, then use ifconfig to
>> create and configure a vlan port I can confirm that the
>> card is
>> working and has connectivity.
>>
>> Here's what I think is happening. ARP requests are received
>> because they are sent to the broadcast address. Our
>> application
>> then announces it's self. However traffic destined for the
>> application is send to a MAC address which is neither the
>> broadcast or the MAC programed into the hardware and is
>> dropped.
>> My understanding of promiscuous it that it informs the
>> card that
>> we want these dropped packets. It looks to me like, when
>> the card
>> is in netmap mode the promisc flag is being ignored.
>>
>> I have also tried using freebsd-update to update to p8. As
>> with
>> the p0 kernel we get a panic when we switch the card into
>> netmap mode.
>>
>> We did previously have these cards working in netmap mode.
>> We were
>> using a pre 11 snapshot of the svn head though .
>>
>> Many Thanks
>>
>> Joe Jones
>> Stream Technologies
>>
>> _______________________________________________
>> freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org>
>> <mailto:freebsd-net at freebsd.org
>> <mailto:freebsd-net at freebsd.org>> mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> <https://lists.freebsd.org/mailman/listinfo/freebsd-net>
>> <https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> <https://lists.freebsd.org/mailman/listinfo/freebsd-net>>
>> To unsubscribe, send any mail to
>> "freebsd-net-unsubscribe at freebsd.org
>> <mailto:freebsd-net-unsubscribe at freebsd.org>
>> <mailto:freebsd-net-unsubscribe at freebsd.org
>> <mailto:freebsd-net-unsubscribe at freebsd.org>>"
>>
>>
>>
>>
>> -- Vincenzo Maffione
>>
>>
>>
>>
>>
>> --
>> Vincenzo Maffione
>>
>
>
--
Vincenzo Maffione
More information about the freebsd-net
mailing list