Random sshd hangs

From: Doug Hardie <bc979_at_lafn.org>
Date: Fri, 14 Jul 2023 03:14:19 UTC
I have a FreeBSD 13.2 system that I use for software development.  I have been working on a project that uses libgd to create charts.  Since libgd is so completely documented, I have to try lots of different things to find the one that will work.  occasionally, during this development, ssh would hang right after running my program.  All existing ssh sessions lock up and eventually time out and disconnect.  However, new ssh sessions can be established and there appears to be nothing unusual with the system.  Top shows everything is quiet.  My first thought was that my code was dumping core, but there are no core files and there is no notice about a core dump in messages.  During this time, web access is also unavailable although the web server is running and showing no arriving connections.

After somewhere around 5-10 minutes, everything returns to normal.  This is not consistent. I can run the program 10 time fine before the next hang.  There is nothing I have found that is common to the hangs.  Now that I have the program running correctly, the system never seems to hang again.  All my ideas on what is causing this are not supported by the facts.  It's not a core dump issue.  It does not appear to be an IP issue as IP continues to work, as do new ssh connections.  The web server is sitting there completely idle.  It never receives any requests.  I suppose it could be something in libgd, but it would seem that would be continuing even now.  I am a lot confused over this.  I don't want to upgrade my production servers as Those hangs would be killers to my clients.  Any ideas what this might be, or what to investigate if it happens again?

-- Doug