fprintf - threadsafe? - i.e. with process linked against '-pthread'?

[Otacílio]

Em 11/10/2017 11:48, Konstantin Belousov escreveu:
> On Wed, Oct 11, 2017 at 10:13:35AM +0100, Karl Pielorz wrote:
>> Hi,
>>
>> I have a number of 10.3-R amd64 boxes which runs a heavily threaded
>> process. This is linked against '-pthread' - and compiles / runs fine.
>>
>> Using 'fprintf' to log data to a file - it sometimes doesn't complete
>> writing the line - e.g. literally in code:
>>
>>     fprintf( fd, "The quick brown %s jumped over the slow lazy animal\n",
>> animal );
>>
>> Will sometimes result in:
>>
>> "
>> The quick brown fox ju"
>>
>> Being written to the file.
>>
>>
>> Presumably (and from what I can see) fprintf is 'thread safe'? - And it
>> also appears multiple threads could write to a single file using it (i.e.
>> it provides for atomic writes so lines won't intermingle - the lines
>> written don't seem to intermingle).
>>
>> The process doesn't crash - but I can't understand why / how frpintf could
>> either stop, or get stopped 'mid way' through?
>>
>> e.g. If a signal occurred would it complete the write to file?
>>
>> This only happens very, very occasionally (one fprintf out of many
>> millions, with hundreds of threads running).
>>
>> Just a bit stumped as to what to try looking at to fix / debug the problem
>> - if anyone has any suggestions, or further reading I can look at.
> Does the program use cancellation ?  If yes, it might be an issue solved
> by the r321074.
>
> Otherwise, you need to debug the program.  I usually use ktrace for start,
> but if the event is rare and program intensively issues syscalls, you would
> need to develop some ad-hoc tecnhique.
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"

This program is able to reproduce the wrong behavior that you report as 
it also contains a solution to it. I tested on a FreeBSD 11.1 running on 
the virtualbox with three processors.

To compile with bug

cc -Wall -O2 -o main main.c -lpthread -DBUGED

./main 10 out

To compile without bug

cc -Wall -O2 -o main main.c -lpthread

./main 10 out

If your problem is something like this, apparently the solution is for 
all threads to use the same variable FILE *

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <pthread.h>

#ifndef MAX_THREADS
#define MAX_THREADS    1000
#endif


void *thread_main(void *file){
     unsigned int i;
     #ifdef BUGED
     FILE *arquivo  = fopen((char*)file, "a");
     #else
     FILE *arquivo = (FILE*) file;
     #endif

     for(i=0; i<3000; i++){
         fprintf(arquivo, "The quick brown %d jumped over the slow lazy 
animal %d\n", getpid(), i);
         pthread_yield();
     }

     pthread_exit(NULL);
     return NULL;
}

int main(int argc, char **argv){
     unsigned int j;
     unsigned int threads;
     static pthread_t mythread[MAX_THREADS];
     FILE *arquivo;

     if(argc != 3){
         fprintf(stderr,"Use %s <numero_threads> <file name>\n", argv[0]);
         exit(EXIT_FAILURE);
     }

     threads = (unsigned int)strtol(argv[1], (char **)NULL, 10);
     if(threads>MAX_THREADS){
         fprintf(stderr, "Max thread suppor %dt\n", MAX_THREADS);
         exit(EXIT_FAILURE);
     }

     if((arquivo = fopen(argv[2],"w+"))==NULL){
         fprintf(stderr, "fopen error\n");
         exit(EXIT_FAILURE);
     }

     for(j=0; j < threads; j++){
         #ifdef BUGED
         if (pthread_create( &mythread[j], NULL, thread_main, argv[2])){
         #else
         if (pthread_create( &mythread[j], NULL, thread_main, arquivo)){
         #endif
             fprintf(stderr,"thread create error.\n");
             exit(EXIT_FAILURE);
         }
     }

     //Vamos esperar pela conclusão de cada um dos threads
     for(j=0; j<threads; j++){
         if ( pthread_join(mythread[j], NULL)) {
             printf("thread join error.\n");
             exit(EXIT_FAILURE);
         }
     }

     fclose(arquivo);
     return 0;
}