cxgbe's native netmap support broken since r307394

Fri Dec 23 01:29:44 UTC 2016

sure go ahead and thank you!

On Thu, Dec 22, 2016 at 5:15 PM, Adrian Chadd <adrian.chadd at gmail.com> wrote:
> ok, does anyone mind if I commit it as-is?
>
>
> -a
>
>
> On 21 December 2016 at 13:37, Vincenzo Maffione <v.maffione at gmail.com> wrote:
>> Hi Luigi,
>>   I attached a minimal change containing two fixes:
>>
>> - change IFNET_WLOCK into IFNET_RLOCK, to fix the cxgbe issue related
>> to this thread
>> - use the proper locking functions for the "worker_lock", unrelated
>> but needed to avoid the O.S. to trap because of a mismatch between
>> MTX_SPIN and MTX_DEF.
>>
>> Cheers,
>>   Vincenzo
>>
>> 2016-12-21 20:30 GMT+01:00 Luigi Rizzo <rizzo at iet.unipi.it>:
>>> On Wed, Dec 21, 2016 at 11:15 AM, Vincenzo Maffione
>>> <v.maffione at gmail.com> wrote:
>>>> Hi,
>>>>   There is no commit related to that in the FreeBSD svn or git.
>>>>
>>>> The fix has been published to the github netmap repository here
>>>> (branch master): https://github.com/luigirizzo/netmap
>>>>
>>>> What we should do is to import all the recent updates from the github
>>>> into HEAD. I can prepare a patch for HEAD, if you wish. Just let me
>>>> know.
>>>
>>> I just checked and the diff between FreeBSD head and netmap head
>>> in github is almost 3k lines due to a lot of recent refactoring.
>>> So, if there is an easy way to extract just the locking change that would
>>> be preferable as an interim solution.
>>>
>>> cheers
>>> luigi
>>>
>>>>
>>>> Cheers,
>>>>   Vincenzo
>>>>
>>>>
>>>> 2016-12-20 21:45 GMT+01:00 Adrian Chadd <adrian.chadd at gmail.com>:
>>>>> hi,
>>>>>
>>>>> What's the commit? We should get it into -HEAD asap.
>>>>>
>>>>>
>>>>> -adrian
>>>>>
>>>>>
>>>>> On 20 December 2016 at 01:25, Vincenzo Maffione <v.maffione at gmail.com> wrote:
>>>>>> Ok, applied to the netmap github repo.
>>>>>> This fix will be published when Luigi does the next commit on FreeBSD.
>>>>>>
>>>>>> Cheers,
>>>>>>   Vincenzo
>>>>>>
>>>>>> 2016-12-19 20:05 GMT+01:00 Navdeep Parhar <np at freebsd.org>:
>>>>>>> IFNET_RLOCK will work, thanks.
>>>>>>>
>>>>>>> Navdeep
>>>>>>>
>>>>>>> On Mon, Dec 19, 2016 at 3:21 AM, Vincenzo Maffione <v.maffione at gmail.com> wrote:
>>>>>>>> Hi Navdeep,
>>>>>>>>
>>>>>>>>   Indeed, we have reviewed the code, and we think it is ok to
>>>>>>>> implement nm_os_ifnet_lock() with IFNET_RLOCK(), instead of using
>>>>>>>> IFNET_WLOCK().
>>>>>>>> Since IFNET_RLOCK() results into sx_slock(), this should fix the issue.
>>>>>>>>
>>>>>>>> On FreeBSD, this locking is needed to protect a flag read by nm_iszombie().
>>>>>>>> However, on Linux the same lock is also needed to protect the call to
>>>>>>>> the nm_hw_register() callback, so we prefer to have an "unified"
>>>>>>>> locking scheme, i.e. always calling nm_hw_register under the lock.
>>>>>>>>
>>>>>>>> Does this make sense to you? Would it be easy for you to make a quick
>>>>>>>> test by replacing IFNET_WLOCK with IFNET_RLOCK?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>   Vincenzo
>>>>>>>>
>>>>>>>> 2016-12-17 23:28 GMT+01:00 Navdeep Parhar <np at freebsd.org>:
>>>>>>>>> Luigi, Vincenzo,
>>>>>>>>>
>>>>>>>>> The last major update to netmap (r307394 and followups) broke cxgbe's
>>>>>>>>> native netmap support.  The problem is that netmap_hw_reg now holds an
>>>>>>>>> rw_lock around the driver's netmap_on/off routines.  It has always been
>>>>>>>>> safe for the driver to sleep during these operations but now it panics
>>>>>>>>> instead.
>>>>>>>>>
>>>>>>>>> Why is IFNET_WLOCK needed here?  It seems like a regression to disallow
>>>>>>>>> sleep on the control path.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Navdeep
>>>>>>>>>
>>>>>>>>> begin_synchronized_op with the following non-sleepable locks held:
>>>>>>>>> exclusive rw ifnet_rw (ifnet_rw) r = 0 (0xffffffff8271d680) locked @
>>>>>>>>> /root/ws/head/sys/dev/netmap/netmap_freebsd.c:95
>>>>>>>>> stack backtrace:
>>>>>>>>> #0 0xffffffff810837a5 at witness_debugger+0xe5
>>>>>>>>> #1 0xffffffff81084d88 at witness_warn+0x3b8
>>>>>>>>> #2 0xffffffff83ef2bcc at begin_synchronized_op+0x6c
>>>>>>>>> #3 0xffffffff83f14beb at cxgbe_netmap_reg+0x5b
>>>>>>>>> #4 0xffffffff809846f1 at netmap_hw_reg+0x81
>>>>>>>>> #5 0xffffffff809806de at netmap_do_regif+0x19e
>>>>>>>>> #6 0xffffffff8098121d at netmap_ioctl+0x7ad
>>>>>>>>> #7 0xffffffff8098682f at freebsd_netmap_ioctl+0x5f
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Vincenzo Maffione
>>>>>>>> _______________________________________________
>>>>>>>> freebsd-net at freebsd.org mailing list
>>>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Vincenzo Maffione
>>>>>> _______________________________________________
>>>>>> freebsd-net at freebsd.org mailing list
>>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>
>>>>
>>>>
>>>> --
>>>> Vincenzo Maffione
>>>
>>>
>>>
>>> --
>>> -----------------------------------------+-------------------------------
>>>  Prof. Luigi RIZZO, rizzo at iet.unipi.it  . Dip. di Ing. dell'Informazione
>>>  http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
>>>  TEL      +39-050-2217533               . via Diotisalvi 2
>>>  Mobile   +39-338-6809875               . 56122 PISA (Italy)
>>> -----------------------------------------+-------------------------------
>>
>>
>>
>> --
>> Vincenzo Maffione

-- 
-----------------------------------------+-------------------------------
 Prof. Luigi RIZZO, rizzo at iet.unipi.it  . Dip. di Ing. dell'Informazione
 http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
 TEL      +39-050-2217533               . via Diotisalvi 2
 Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------