kevent behavior with TCP socket

[Denis Berezhnoy]

Hi Adrian,
Thank you very much for clarification! Now it is clear for me.  I have the
last question.

When I try to make mulltiple connections (loop over 100 client sockets) and
check status of sockets by getsockopt then for the first couple connections
 everything seems OK, no error code but for the following connections I got
socket error 54 ECONNRESET.

I am concerned about it  since I expected to get ECONNREFUSED.

Here is code snippet (client thread the server part which listens socket is
the same as in my previous example) and output


void *ThreadFunc(void *arg)
{
    struct sockaddr_in addr2;
    int cd[100], port;

    int sRet, i;

    int  sHandle;
    int  sEventNum = 0;

    struct kevent     sChange;
    struct kevent     sEvent;

    struct timespec  *sTimeoutPtr;
    struct timespec   sTimeout;

    struct timeval sTimeVal1 = {0,0};
    struct timeval sTimeVal2 = {0,0};

 for (i=0;i < 100; i++)
 {
    cd[i] = socket(AF_INET, SOCK_STREAM, 0);

    if ( cd[i] < 0 )
    {
         printf("Client socket error\n");
         return 0;
    }

    sRet = fcntl(cd[i], F_GETFL, 0);

    sRet |= O_NONBLOCK;

    sRet = fcntl(cd[i], F_SETFL, sRet);

    if (sRet < 0)
    {
        printf("can not set non block\n");
    }

    port = htons(10000);

    memset(&addr2, 0, sizeof(addr2));       /* Clear struct */
    addr2.sin_family = AF_INET;                  /* Internet/IP */
    addr2.sin_addr.s_addr = htonl(INADDR_LOOPBACK);  /* IP address */
    addr2.sin_port = port;       /* server port */

    /* Establish connection */

    if ((sRet = connect(cd[i],
                (struct sockaddr *) &addr2,
                sizeof(addr2))) < 0)
    {
        printf("Failed to connect with server %d\n", errno);
    }

    sHandle = kqueue();

    if (sHandle == -1)
    {
         printf("Poll can not created queue\n");
    }

    sTimeout.tv_sec  = 0;
    sTimeout.tv_nsec = 100000000;;

    EV_SET(&sChange, cd[i], EVFILT_WRITE, EV_ADD,0, 0, 0);

    gettimeofday(&sTimeVal1, NULL);

    sEventNum = kevent(sHandle,
                       &sChange,
                       1,
                       &sEvent,
                       1,
                       &sTimeout);

    gettimeofday(&sTimeVal2, NULL);

    printf ("Kevent event num %d wait time %d \n", sEventNum,
sTimeVal2.tv_usec - sTimeVal1.tv_usec);

    if (sEventNum == 1)
    {
        int aOptVal = 0;
        int aOptLen = sizeof(aOptVal);

        printf ("Event filter %d flag %d filter flags %d data %d \n",
sEvent.filter, sEvent.flags, sEvent.fflags, sEvent.data);

        sRet = getsockopt(cd[i], SOL_SOCKET, SO_ERROR, &aOptVal, &aOptLen);

        printf("error %d sockopt ret %d\n", aOptVal, sRet);
    }

      close(sHandle);
      close(cd[i]);
   }

   pthread_exit(NULL);
}



OutputL

Socket test start
Failed to connect with server 36
Kevent event num 1 wait time 24
Event filter -2 flag 0 filter flags 0 data 43008
error 0 sockopt ret 0
Failed to connect with server 36
Kevent event num 1 wait time 21
Event filter -2 flag 0 filter flags 0 data 43008
error 0 sockopt ret 0
Failed to connect with server 36
Kevent event num 1 wait time 13
Event filter -2 flag 0 filter flags 0 data 43008
error 54 sockopt ret 0
Failed to connect with server 36
Kevent event num 1 wait time 20
Event filter -2 flag 0 filter flags 0 data 43008
error 54 sockopt ret 0


Best regards,
Denis

2009/8/10 Adrian Penisoara <ady at freebsd.ady.ro>

> Hi,
>
> On Mon, Aug 10, 2009 at 5:17 AM, Denis Berezhnoy <
> denis.berezhnoy at gmail.com> wrote:
>
>> Hi Adrian,
>> Thank for your answer! I checked that nobody listens for loopback address:
>>
>> [denis at freebsd ~]$ sockstat -4
>> USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
>>
>> denis    sshd       95390 3  tcp4   192.168.1.103:22
>> 192.168.11.26:53616
>> root     sshd       95387 3  tcp4   192.168.1.103:22
>> 192.168.11.26:53616
>> denis    sshd       95324 3  tcp4   192.168.1.103:22
>> 192.168.11.26:53608
>> root     sshd       95321 3  tcp4   192.168.1.103:22
>> 192.168.11.26:53608
>> root     sshd       8149  4  tcp4   *:22                  *:*
>> root     inetd      870   5  tcp4   *:21                  *:*
>> root     inetd      870   6  tcp4   *:23                  *:*
>> root     sendmail   821   4  tcp4   127.0.0.1:25          *:*
>> root     syslogd    689   7  udp4   *:514                 *:*
>> [denis at freebsd ~]$
>>
>>
>> I printed out fflags. Here is the result:
>>
>> Kevent event num 1 wait time 46
>> Event filter -2 flag 0 filter flags 0 data 43008
>>
>>
>> When I comment out the part of the code that listens socket then I have
>> the following output:
>>
>> Kevent event num 1 wait time 23
>> Event filter -2 flag 32768 filter flags 61 data 32768
>>
>>
>> 61 is ECONNREFUSED
>>
>>
>> And more when I make loopback address listening  and try to connect it I
>> still get the same output:
>>
>> Socket test start
>> Failed to connect with server 36
>> Kevent event num 1 wait time 23
>> Event filter -2 flag 0 filter flags 0 data 43008
>> Socket test end
>>
>>  but connection is actually accepted. I am totally confused here.
>>
>
> Well, actually it's very much evident:
>  * When you don't make the listen() first, then the connection is correctly
> reported as refused
>  * When you make the listen() on the INADDR_ANY then you will open port
> 10000 on all local IP addresses, _including_ the loopback address -- that's
> why the client connect() succeeds and kevent reports a success event. This
> is an expected behavior.
>
>  Try to bind the sd descriptor to another port than the one used to connect
> onto the loopback address and you will see the connection being refused.
>
> Regards,
> Adrian
> EnterpriseBSD
>