Help needed to identify golang fork / memory corruption issue on FreeBSD

[Steven Hartland]

On 28/03/2017 09:38, Konstantin Belousov wrote:
> On Tue, Mar 28, 2017 at 09:23:24AM +0100, Steven Hartland wrote:
>> As I stopped the panic before that I couldn't tell so I've re-run with
>> some debug added just before the panic to capture the addresses of the
>> workbuf structure that the issue was detected in, here goes (parent:
>> 62620, child: 98756):
>>
>> workbuf: 0x800b51800
>> fatal error: workbuf is not empty
>> workbuf: 0x800a72000
>> fatal error: workbuf is empty
>> workbuf: 0x800a72000
>> fatal error: workbuf is not empty
> I do not understand.  Why do you show several addresses ?  Wouldn't the
> runtime panic after detecting the discrepancy, so there could be only one
> address ?
There are several goroutines (threads) running each detected an error, 
as I'm blocking the panic by entering a sleep in the faulting goroutine 
to enable the capture of procstat, other routines continue and detect an 
error too.

To be clear this is not exclusive to the addition of the sleep, as I've 
seen this without any changes, as the GC is multi threaded.

>
>> procstat -v 62620 98756
>>     PID              START                END PRT  RES PRES REF SHD FLAG
>> TP PATH
>> 62620           0x400000           0x70e000 r-x  309  595 5   1 CN-- vn
>> /root/golang/src/test5/test5
>> 62620           0x70e000           0x951000 r--  254  595 5   1 CN-- vn
>> /root/golang/src/test5/test5
>> 62620           0x951000           0x988000 rw-   31    0 1   0 C--- vn
>> /root/golang/src/test5/test5
>> 62620           0x988000           0x9ab000 rw-   18    0 1   0 C--- df
>> 62620        0x800951000        0x8009f1000 rw-   36    0 1   0 C--- df
>> 62620        0x8009f1000        0x800ac1000 rw-   28    0 1   0 C--- df
>> 62620        0x800ac1000        0x800ad1000 rw-   16    0 1   0 C--- df
>> 62620        0x800ad1000        0x800b91000 rw-    5    0 1   0 C--- df
>> 62620        0x800b91000        0x800bd1000 rw-    2    0 1   0 C--- df
>> 62620        0x800bd1000        0x800c11000 rw-    2    0 1   0 C--- df
>> 62620        0x800c11000        0x800c51000 rw-    1    1 2   0 CN-- df
>> 62620        0x800c51000        0x800c91000 rw-    1    0 1   0 C--- df
>> 62620        0x800c91000        0x800cd1000 rw-    1    0 1   0 C--- df
>> 62620       0xc000000000       0xc000001000 rw-    1    1 2   0 CN-- df
>> 62620       0xc41fff0000       0xc41fff8000 rw-    3    3 2   0 CN-- df
>> 62620       0xc41fff8000       0xc420200000 rw-  251    0 1   0 C--- df
>> 62620     0x7ffffffdf000     0x7ffffffff000 rwx    2    0 1   0 C--D df
>> 62620     0x7ffffffff000     0x800000000000 r-x    1    1 28   0 ---- ph
>> 98756           0x400000           0x70e000 r-x  309  595 5   1 CN-- vn
>> /root/golang/src/test5/test5
>> 98756           0x70e000           0x951000 r--  254  595 5   1 CN-- vn
>> /root/golang/src/test5/test5
>> 98756           0x951000           0x988000 rw-   31    0 2   1 CN-- vn
>> /root/golang/src/test5/test5
>> 98756           0x988000           0x9ab000 rw-   18   18 2   1 CN-- df
>> 98756        0x800951000        0x8009f1000 rw-   36   36 2   1 CN-- df
>> 98756        0x8009f1000        0x800ac1000 rw-   28   28 2   1 CN-- df
>> 98756        0x800ac1000        0x800ad1000 rw-   16   16 2   1 CN-- df
>> 98756        0x800ad1000        0x800b91000 rw-    5    5 2   1 CN-- df
>> 98756        0x800b91000        0x800bd1000 rw-    2    2 2   1 CN-- df
>> 98756        0x800bd1000        0x800c11000 rw-    2    2 2   1 CN-- df
>> 98756        0x800c11000        0x800c51000 rw-    1    1 2   0 CN-- df
>> 98756        0x800c51000        0x800c91000 rw-    1    1 2   1 CN-- df
>> 98756        0x800c91000        0x800cd1000 rw-    1    1 2   1 CN-- df
>> 98756       0xc000000000       0xc000001000 rw-    1    1 2   0 CN-- df
>> 98756       0xc41fff0000       0xc41fff8000 rw-    3    3 2   0 CN-- df
>> 98756       0xc41fff8000       0xc420200000 rw-  251    0 1   0 C--- df
>> 98756     0x7ffffffdf000     0x7ffffffff000 rwx    2    2 2   1 CN-D df
>> 98756     0x7ffffffff000     0x800000000000 r-x    1    1 28   0 ---- ph
>>
>>       Regards
>>       Steve