Help needed to identify golang fork / memory corruption issue on FreeBSD

[Konstantin Belousov]

On Mon, Mar 27, 2017 at 12:47:11PM +0100, Steven Hartland wrote:
> OK now the similar but unrelated issue with signal stacks is solved I've 
> moved back to the initial issue.
> 
> I've made some progress with a reproduction case as detailed here:
> https://github.com/golang/go/issues/15658#issuecomment-288747812
> 
> In short it seems that having a running child, while the parent runs GC, 
> is some how responsible for memory corruption in the parent.
> 
> The reason I believe this is if I run the same GC in the parent after 
> the child exits instead of while its running, I've been unable to 
> reproduce the issue.
> 
> As the memory segments are COW then the issue might be in VM subsystem.
Well, it might be, but it is a strange corruption mode to believe.

> 
> In order to confirm / deny this I was wondering if there was a way to 
> force a full copy of all segments for the child instead of using the COW 
> optimisation.
No, there is no. By design, copying only occurs on faults, when VM
detects that the map entry needs copying. Doing the actual copy at fork
time would require writing a lot of new code.

Does go have FreeBSD/i386 port ?  If yes, is the issue reproducable there ?

Another blind experiment to try is to comment out call to
vm_object_collapse() in sys/vm/vm_map.c:vm_map_copy_entry() and see if
it changes anything.

What could be quite interesting is to look at the parent and possibly
child address map after the error occured, using procstat -v. At
least for parent, this should be relatively easy to set up, just make
go runtime spin or pause on panic, instead of exiting, and then use
procstat.

> 
> Is this something that would be relatively easy to hack into the kernel, 
> and if so pointers would be appreciated.

BTW, I looked some more at the go code, and I noted that
runtime<stupid UTF-8 char>mmap() implementation looks very strange.
It ignores %rflags.C bit to identify error, and instead callers
of mmap() compare the return value with 4096, assuming Linux-style
error reporting.  This would certainly break if mmap(2) syscall
returns ERESTART one day.