cxgbe's native netmap support broken since r307394

[Adrian Chadd]

ok, does anyone mind if I commit it as-is?


-a


On 21 December 2016 at 13:37, Vincenzo Maffione <v.maffione at gmail.com> wrote:
> Hi Luigi,
>   I attached a minimal change containing two fixes:
>
> - change IFNET_WLOCK into IFNET_RLOCK, to fix the cxgbe issue related
> to this thread
> - use the proper locking functions for the "worker_lock", unrelated
> but needed to avoid the O.S. to trap because of a mismatch between
> MTX_SPIN and MTX_DEF.
>
> Cheers,
>   Vincenzo
>
> 2016-12-21 20:30 GMT+01:00 Luigi Rizzo <rizzo at iet.unipi.it>:
>> On Wed, Dec 21, 2016 at 11:15 AM, Vincenzo Maffione
>> <v.maffione at gmail.com> wrote:
>>> Hi,
>>>   There is no commit related to that in the FreeBSD svn or git.
>>>
>>> The fix has been published to the github netmap repository here
>>> (branch master): https://github.com/luigirizzo/netmap
>>>
>>> What we should do is to import all the recent updates from the github
>>> into HEAD. I can prepare a patch for HEAD, if you wish. Just let me
>>> know.
>>
>> I just checked and the diff between FreeBSD head and netmap head
>> in github is almost 3k lines due to a lot of recent refactoring.
>> So, if there is an easy way to extract just the locking change that would
>> be preferable as an interim solution.
>>
>> cheers
>> luigi
>>
>>>
>>> Cheers,
>>>   Vincenzo
>>>
>>>
>>> 2016-12-20 21:45 GMT+01:00 Adrian Chadd <adrian.chadd at gmail.com>:
>>>> hi,
>>>>
>>>> What's the commit? We should get it into -HEAD asap.
>>>>
>>>>
>>>> -adrian
>>>>
>>>>
>>>> On 20 December 2016 at 01:25, Vincenzo Maffione <v.maffione at gmail.com> wrote:
>>>>> Ok, applied to the netmap github repo.
>>>>> This fix will be published when Luigi does the next commit on FreeBSD.
>>>>>
>>>>> Cheers,
>>>>>   Vincenzo
>>>>>
>>>>> 2016-12-19 20:05 GMT+01:00 Navdeep Parhar <np at freebsd.org>:
>>>>>> IFNET_RLOCK will work, thanks.
>>>>>>
>>>>>> Navdeep
>>>>>>
>>>>>> On Mon, Dec 19, 2016 at 3:21 AM, Vincenzo Maffione <v.maffione at gmail.com> wrote:
>>>>>>> Hi Navdeep,
>>>>>>>
>>>>>>>   Indeed, we have reviewed the code, and we think it is ok to
>>>>>>> implement nm_os_ifnet_lock() with IFNET_RLOCK(), instead of using
>>>>>>> IFNET_WLOCK().
>>>>>>> Since IFNET_RLOCK() results into sx_slock(), this should fix the issue.
>>>>>>>
>>>>>>> On FreeBSD, this locking is needed to protect a flag read by nm_iszombie().
>>>>>>> However, on Linux the same lock is also needed to protect the call to
>>>>>>> the nm_hw_register() callback, so we prefer to have an "unified"
>>>>>>> locking scheme, i.e. always calling nm_hw_register under the lock.
>>>>>>>
>>>>>>> Does this make sense to you? Would it be easy for you to make a quick
>>>>>>> test by replacing IFNET_WLOCK with IFNET_RLOCK?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>   Vincenzo
>>>>>>>
>>>>>>> 2016-12-17 23:28 GMT+01:00 Navdeep Parhar <np at freebsd.org>:
>>>>>>>> Luigi, Vincenzo,
>>>>>>>>
>>>>>>>> The last major update to netmap (r307394 and followups) broke cxgbe's
>>>>>>>> native netmap support.  The problem is that netmap_hw_reg now holds an
>>>>>>>> rw_lock around the driver's netmap_on/off routines.  It has always been
>>>>>>>> safe for the driver to sleep during these operations but now it panics
>>>>>>>> instead.
>>>>>>>>
>>>>>>>> Why is IFNET_WLOCK needed here?  It seems like a regression to disallow
>>>>>>>> sleep on the control path.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Navdeep
>>>>>>>>
>>>>>>>> begin_synchronized_op with the following non-sleepable locks held:
>>>>>>>> exclusive rw ifnet_rw (ifnet_rw) r = 0 (0xffffffff8271d680) locked @
>>>>>>>> /root/ws/head/sys/dev/netmap/netmap_freebsd.c:95
>>>>>>>> stack backtrace:
>>>>>>>> #0 0xffffffff810837a5 at witness_debugger+0xe5
>>>>>>>> #1 0xffffffff81084d88 at witness_warn+0x3b8
>>>>>>>> #2 0xffffffff83ef2bcc at begin_synchronized_op+0x6c
>>>>>>>> #3 0xffffffff83f14beb at cxgbe_netmap_reg+0x5b
>>>>>>>> #4 0xffffffff809846f1 at netmap_hw_reg+0x81
>>>>>>>> #5 0xffffffff809806de at netmap_do_regif+0x19e
>>>>>>>> #6 0xffffffff8098121d at netmap_ioctl+0x7ad
>>>>>>>> #7 0xffffffff8098682f at freebsd_netmap_ioctl+0x5f
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Vincenzo Maffione
>>>>>>> _______________________________________________
>>>>>>> freebsd-net at freebsd.org mailing list
>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Vincenzo Maffione
>>>>> _______________________________________________
>>>>> freebsd-net at freebsd.org mailing list
>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>
>>>
>>>
>>> --
>>> Vincenzo Maffione
>>
>>
>>
>> --
>> -----------------------------------------+-------------------------------
>>  Prof. Luigi RIZZO, rizzo at iet.unipi.it  . Dip. di Ing. dell'Informazione
>>  http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
>>  TEL      +39-050-2217533               . via Diotisalvi 2
>>  Mobile   +39-338-6809875               . 56122 PISA (Italy)
>> -----------------------------------------+-------------------------------
>
>
>
> --
> Vincenzo Maffione