threads/148515: Memory / syslog strangeness in FreeBSD 8.x ( possible leak/ threading issue )

[Vikash]

>Number:         148515
>Category:       threads
>Synopsis:       Memory / syslog strangeness in FreeBSD 8.x ( possible leak/ threading issue )
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-threads
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Jul 12 08:10:05 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Vikash
>Release:        8.0 RELEASE
>Organization:
>Environment:
FreeBSD vix.isnet.net 8.0-RELEASE-p3 FreeBSD 8.0-RELEASE-p3 #11: Sun Jun 27 20:10:59 SAST 2010     root at vix.isnet.net:/usr/obj/usr/src/sys/GENERIC  i386

>Description:
Can someone look into this strangeness on FreeBSD 8.0 and 8.1-RC2

using a threaded test code, I see the that freebsd 8.x seems to be using more memory than freebsd 7.x:


Results:

7.2
without syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
26872 vikashb    1001   8    0   128M 14236K RUN      0:00  0.00% a.out

with syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
26881 vikashb    1001  44    0   128M 26236K RUN      0:00  0.00% a.out

8.0-RELEASE-p3
without syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
61529 vikashb    1001  44    0   129M 14840K RUN      0:01  0.00% a.out

with syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
61507 vikashb    1001  44    0   257M 42708K RUN      0:30  0.00% a.out

8.1-RC2
without syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
33062 vikashb    1001  44    0   129M 14804K RUN      0:00  0.00% a.out

with syslog
  PID USERNAME    THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
33056 vikashb    1001  44    0   257M 42708K RUN      0:03  0.00% a.out


I have not been able to find any reasonable information via Google/ PR search


either there is a leak or there is an undocumented behaviour in the 8.x threads.

neither freebsd-questions at freebsd.org nor freebsd-threads at freebsd.org yielded ant responses.

>How-To-Repeat:
compile the code below and observe the memory usage

8.x uses more memory than 7.x

<CODE>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#include <stdarg.h>
#include <errno.h>
#include <syslog.h>
#include <signal.h>
#include <pthread.h>

char *ProgramName = "WTF";

int loop = 0;
int LogToSTDOUT = 1;

void LogMessage(int debug, const char *fmt,...) {
   extern int LogToSTDOUT;

   char message[8192];

   memset(message, 0, sizeof(message));

   va_list args;

   va_start(args, fmt);
   vsnprintf(message, sizeof(message), fmt, args);
   va_end(args);

   if ( LogToSTDOUT )
   {
      printf("%s\n", message);
   }

   syslog(LOG_NOTICE, "%s", message);

}

unsigned long int getTimeNow()
{
   struct timeval tv;

   if ( gettimeofday(&tv, NULL) == -1 )
   {
      LogMessage(0, "ERROR(%d) %s\n", errno, strerror(errno));
      tv.tv_sec = 0;
   }

   return tv.tv_sec;
}

void HandleSignal(int sig)
{
   loop = 0;
   LogMessage(0, "loop  = %d\n", loop);
   signal(sig, SIG_IGN);
   usleep(1000);
}

void *worker(int n)
{
   while ( loop )
   {
      LogMessage(0, "worker #%d logging", n);
      usleep(1000);
   }

   pthread_exit(0);
}

int main(int argc, char* argv[])
{
   pthread_t* tpool;

   int workers = 1000, i, rc;

   openlog(ProgramName, LOG_PID, LOG_MAIL);

   unsigned long int duration = 120, StartTime, TimeNow;

   signal(SIGINT, HandleSignal);
   signal(SIGTERM, HandleSignal);
   signal(SIGHUP, HandleSignal);
   signal(SIGQUIT, HandleSignal);

   StartTime = getTimeNow();

   tpool = (pthread_t*)malloc(workers * sizeof(pthread_t));

   if ( tpool == NULL )
   {
      LogMessage(0, "malloc failed \n");
      closelog();
      exit(-1);
   }

   memset(tpool, 0, sizeof(pthread_t) * workers);

   loop = 1;

   for ( i = 0; i < workers; i++ )
   {
      rc = pthread_create(&tpool[i], NULL, (void *(*)(void*))&worker, (void*)i);

      if ( rc != 0 )
      {
         LogMessage(0, "pthread_create #%d failed\n", i );
         pthread_cancel(tpool[i]);
      }
      else
      {
         pthread_detach(tpool[i]);
      }
   }

   LogMessage(0, "loop  = %d\n", loop);

   while ( loop )
   {
      TimeNow = getTimeNow();

      if ( ( TimeNow - StartTime )  > duration )
      {
          loop = 0;
      }

      usleep(1000);
   }

   for ( i = 0; i < workers; i++ )
   {
      pthread_cancel(tpool[i]);
   }

   for ( i = 0; i < workers; i++ )
   {
      pthread_join(tpool[i], NULL);
   }

   free(tpool);
   closelog();
   exit(0);
}

</CODE>

>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: