cxgbe netmap promiscuous mode?
Joe Jones
joe at stream-technologies.com
Fri Mar 24 13:50:49 UTC 2017
Hi Vincenzo,
we can get rid if the panic if we compile the kernel without netmap,
then load an up to date netmap as a module. We can live with that for now.
Some time last year before Free BSD 11 was branched from head, we
compiled a checkout and it worked without problem on the same hardware
setup. I may try and figure out what the regression is, I'm not familiar
with the FreeBSD release process either though.
Thanks,
Joe Jones
On 24/03/17 10:29, Vincenzo Maffione wrote:
> Hi Joe,
> There was a fix for a panic in emulated mode that was applied
> stable/11 branch, so I guess it also ended up into FreeBSD-11.0-STABLE.
> I don't know whether the same fix ended up into in 11.0-RELEASE-p8
> (I'm not familiar with FreeBSD releasing process, sorry!).
>
> Or maybe this panic happens with netmap upstream?
> If this is a new bug, it would be nice to have the kernel with the
> debug symbols enabled, so that we can get more detailed information
> from the stack trace.
>
> Cheers,
> Vincenzo
>
> 2017-03-24 11:19 GMT+01:00 Joe Jones <joe at stream-technologies.com
> <mailto:joe at stream-technologies.com>>:
>
> Hi Vincenzo,
>
> I just tried with that sysctl set to 2, I get a similar looking
> panic to before
>
> Fatal trap 12: page fault while in kernel mode
> cpuid = 7; apic id = 0e
> fault virtual address = 0x1
> fault code = supervisor read data, page not present
> instruction pointer = 0x20:0xffffffff806e38aa
> stack pointer = 0x28:0xfffffe047ba18440
> frame pointer = 0x28:0xfffffe047ba18490
> code segment = base 0x0, limit 0xfffff, type 0x1b
> = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags = interrupt enabled, resume, IOPL = 0
> current process = 2205 (main)
> trap number = 12
> panic: page fault
> cpuid = 7
> KDB: stack backtrace:
> #0 0xffffffff80b240f7 at kdb_backtrace+0x67
> #1 0xffffffff80ad9462 at vpanic+0x182
> #2 0xffffffff80ad92d3 at panic+0x43
> #3 0xffffffff80fa1d51 at trap_fatal+0x351
> #4 0xffffffff80fa1f43 at trap_pfault+0x1e3
> #5 0xffffffff80fa14ec at trap+0x26c
> #6 0xffffffff80f841c1 at calltrap+0x8
> #7 0xffffffff806e5a80 at generic_netmap_txsync+0x330
> #8 0xffffffff806e06f9 at netmap_ioctl+0x279
> #9 0xffffffff8098624f at devfs_ioctl_f+0x13f
> #10 0xffffffff80b41b34 at kern_ioctl+0x2d4
> #11 0xffffffff80b417f1 at sys_ioctl+0x171
> #12 0xffffffff80fa26ae at amd64_syscall+0x4ce
> #13 0xffffffff80f844ab at Xfast_syscall+0xfb
>
> This is on 11.0-RELEASE-p8
>
> Thanks,
> Joe Jones
>
> On 23/03/17 18:20, Vincenzo Maffione wrote:
>
> Hi,
> You could try to use netmap in emulated mode (sysctl
> dev.netmap.admode=2). If this works, at least you know that
> the problem is in the cxgbe netmap support and not in the
> netmap core itself.
>
> Cheers,
> Vincenzo
>
> 2017-03-23 14:00 GMT+01:00 Joe Jones
> <joe at stream-technologies.com
> <mailto:joe at stream-technologies.com>
> <mailto:joe at stream-technologies.com
> <mailto:joe at stream-technologies.com>>>:
>
> Hello,
>
> We have a T520-SO and have made a new install of 11.0, to
> begin
> with the box would panic every time we tried to switch the
> card
> into netmap mode. So we recompiled the kernel with netmap
> removed,
> then compiled the netmap kernel module from github, as
> this in our
> experience generally leads to a more stable netmap.
>
> we have
>
> uname -a
> FreeBSD goose2 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0:
> Wed Mar
> 22 16:52:35 UTC 2017
> joe at goose2:/usr/obj/usr/src/sys/GENERIC amd64
>
> and the following in /boot/loader.conf
>
> t4fw_cfg_load="YES"
> t5fw_cfg_load="YES"
> if_cxgbe_load="YES"
> hw.cxgbe.fl_pktshift=0
> hw.cxgbe.toecaps_allowed=0
> hw.cxgbe.nnmtxq10g=8
> hw.cxgbe.nnmrxq10g=8
> hw.cxgbe.num_vis=2
>
> Before I run our application I run
>
> ifconfig cxl1 promisc -vlanhwtag up
>
> Now our application can now start without panicking the
> kernel.
> Here is where it gets interesting, our application is able to
> announce it's self via ARP, I can see the ethernet switch
> learning
> which port it's on, and other hosts adding it to their ARP
> tables.
> When I try an ICMP ping it goes missing. After watching the TX
> packet graph for the connected port on the switch while
> starting
> and stopping a flood ping to the application, I'm sure the
> packets
> are getting sent to the card, however I don't see them in the
> netmap ring. If I kill our application, then use ifconfig to
> create and configure a vlan port I can confirm that the
> card is
> working and has connectivity.
>
> Here's what I think is happening. ARP requests are received
> because they are sent to the broadcast address. Our
> application
> then announces it's self. However traffic destined for the
> application is send to a MAC address which is neither the
> broadcast or the MAC programed into the hardware and is
> dropped.
> My understanding of promiscuous it that it informs the
> card that
> we want these dropped packets. It looks to me like, when
> the card
> is in netmap mode the promisc flag is being ignored.
>
> I have also tried using freebsd-update to update to p8. As
> with
> the p0 kernel we get a panic when we switch the card into
> netmap mode.
>
> We did previously have these cards working in netmap mode.
> We were
> using a pre 11 snapshot of the svn head though .
>
> Many Thanks
>
> Joe Jones
> Stream Technologies
>
> _______________________________________________
> freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org>
> <mailto:freebsd-net at freebsd.org
> <mailto:freebsd-net at freebsd.org>> mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> <https://lists.freebsd.org/mailman/listinfo/freebsd-net>
> <https://lists.freebsd.org/mailman/listinfo/freebsd-net
> <https://lists.freebsd.org/mailman/listinfo/freebsd-net>>
> To unsubscribe, send any mail to
> "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>
> <mailto:freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>>"
>
>
>
>
> --
> Vincenzo Maffione
>
>
>
>
>
> --
> Vincenzo Maffione
More information about the freebsd-net
mailing list