FreeBSD 8.1-PRERELEASE: WARNING ioctl sign-extension ioctl ffffffff8004667e

Sat Jun 26 09:13:02 UTC 2010

On 25/06/2010 18:58, Jung-uk Kim wrote:
> On Friday 25 June 2010 04:54 am, Mario Sergio Fujikawa Ferreira wrote:
>> On Wed, Jun 23, 2010 at 05:08:38PM -0400, Jung-uk Kim wrote:
>>> Date: Wed, 23 Jun 2010 17:08:38 -0400
>>> From: Jung-uk Kim<jkim at FreeBSD.org>
>>> To: freebsd-stable at FreeBSD.org
>>> Cc: d at delphij.net, Mario Sergio Fujikawa Ferreira
>>> <lioux at freebsd.org>  Subject: Re: FreeBSD 8.1-PRERELEASE: WARNING
>>> ioctl sign-extension ioctl ffffffff8004667e
>>> User-Agent: KMail/1.6.2
>>>
>>> On Wednesday 23 June 2010 03:38 pm, Jung-uk Kim wrote:
>>>> On Wednesday 23 June 2010 03:10 pm, Jung-uk Kim wrote:
>>>>> On Wednesday 23 June 2010 03:01 pm, Jung-uk Kim wrote:
>>>>>> On Wednesday 23 June 2010 02:41 pm, Xin LI wrote:
>>>>>>> On 2010/06/23 11:37, Jung-uk Kim wrote:
>>>>>>>> On Wednesday 23 June 2010 02:12 pm, Xin LI wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On 2010/06/22 19:58, Mario Sergio Fujikawa Ferreira
> wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> 	I am getting more than 4 thousand of the following
>>>>>>>>>> messages a day:
>>>>>>>>>>
>>>>>>>>>> WARNING pid 24509 (python2.6): ioctl sign-extension
>>>>>>>>>> ioctl ffffffff8004667e
>>>>>>>>>
>>>>>>>>> [...]
>>>>>>>>>
>>>>>>>>> I think we may need to check the code and patch it.
>>>>>>>>> Basically this means that python (or some .so modules)
>>>>>>>>> passed an int or unsigned int as parameter 'cmd', we
>>>>>>>>> need to change it  to unsigned long.
>>>>>>>>>
>>>>>>>>> The warning itself should be harmless to my best of
>>>>>>>>> knowledge, one can probably remove the printf in
>>>>>>>>> kernel source code as a workaround.
>>>>>>>>>
>>>>>>>>> By the way it seems to be a POSIX violation and we
>>>>>>>>> didn't seem to really use so wide cmd, but I have not
>>>>>>>>> yet verified everything myself.
>>>>>>>>
>>>>>>>> Long time ago, I had a similar problem with termios
>>>>>>>> TIOCGWINSZ and we patched the port like this:
>>>>>>>>
>>>>>>>> http://www.freebsd.org/cgi/cvsweb.cgi/ports/lang/python
>>>>>>>> /fil es /A tt
>>>>>>>> ic/patch-Modules%3A%3Afcntlmodule.c?rev=1.1;content-typ
>>>>>>>> e=te xt %2 Fpl ain
>>>>>>>>
>>>>>>>> I believe it was upstream patched at the time but I
>>>>>>>> won't be surprised if something similar was
>>>>>>>> reintroduced.  It happens when a Python internal
>>>>>>>> integer type is converted to a native unsigned long.
>>>>>>>
>>>>>>> Well, only *BSD have cmd a long value so it's likely that
>>>>>>> it would be reintroduced.
>>>>>>
>>>>>> Yes, that's what I mean.
>>>>>>
>>>>>>> I have checked the 4.4BSD archive and understood that our
>>>>>>> ioctl's cmd parameter was made long around 1991 or 1992s
>>>>>>> but didn't see what it actually buy us...
>>>>>>
>>>>>> Like it or not, it is too late to revert. :-(
>>>>>
>>>>>  From Python 2.6.5:
>>>>>
>>>>> static PyObject *
>>>>> fcntl_ioctl(PyObject *self, PyObject *args)
>>>>> {
>>>>> #define IOCTL_BUFSZ 1024
>>>>> 	int fd;
>>>>> 	/* In PyArg_ParseTuple below, we use the unsigned
>>>>> non-checked 'I' format for the 'code' parameter because
>>>>> Python turns 0x8000000 into either a large positive number
>>>>> (PyLong or PyInt on 64-bit platforms) or a negative number on
>>>>> others (32-bit PyInt) whereas the system expects it to be a
>>>>> 32bit bit field value regardless of it being passed as an int
>>>>> or unsigned long on various platforms.  See the
>>>>> termios.TIOCSWINSZ constant across platforms for an example
>>>>> of thise.
>>>>>
>>>>> 	   If any of the 64bit platforms ever decide to use more
>>>>> than 32bits in their unsigned long ioctl codes this will
>>>>> break and need special casing based on the platform being
>>>>> built on. */
>>>>> 	unsigned int code;
>>>>> ...
>>>>>
>>>>> It is still a kludge and it won't be fixed. :-(
>>>>
>>>> Please drop the attached patch in ports/lang/python26/files and
>>>> test. Note I am not a Python guy, so please don't shoot me. ;-)
>>>
>>> I found that Python guys added 'k' format since 2.3, which
>>> converts a Python integer to unsigned long.  Please ignore the
>>> previous patch and try the attached patch instead.
>>
>> 	Unfortunately it didn't work.
>>
>> 	1) Added patch to lang/python26
>> 	2) Recompiled lang/python26
>> 	3) cd /var/db/ports&&  portupgrade -f python* py26* deluge*
>>
>> 	Restarted deluge afterwards. The message is still there:
>>
>> Jun 25 05:49:38 exxodus kernel: WARNING pid 38635 (python2.6):
>> ioctl sign-extension ioctl ffffffff8004667e
>
> It may be a deeper problen, then. :-(
>
> First of all, I cannot reproduce the problem because deluged dumps
> core on my box and I gave it up.  Just staring at the code, it seems
> they rely on a bundled libtorrent for Python binding and the
> libtorrent relies on Boost, in turn.  Then, the actual non-blocking
> socket implementation is done with Boost ASIO[1].  It is possible
> that there are more complicated problems between these and the Python
> binding.  In fact, I can see a lot of these:
>
>    int name() const
>    {
>      return FIONBIO;
>    }
> ...
>    if (!ec&&  command.name() == static_cast<int>(FIONBIO))
> ...
>    socket_ops::ioctl(impl.socket_, command.name(), ...)
> ...
>
> They are just assuming it is an int type, I guess.
>
> Sigh, what a mess...

	Given that your python patch is a step on the right direction, I would 
propose that it be further tested and then committed if possible.

> [1] Boost and its Python binding from ports seems to be a newer ASIO
> version than the bundled ASIO headers.  Probably it is a reason for
> the crash but I didn't bother too much.

	If you have the time, everything you need to run the latest deluge 
1.3.0 RC1 can be found at

http://www.freebsd.org/cgi/query-pr.cgi?pr=144337

	Check the PR, all the necessary ports can be found there. Let me know 
what you think.

	Regards,

--
Mario S F Ferreira - DF - Brazil - "I guess this is a signature."
feature, n: a documented bug | bug, n: an undocumented feature