Re: SSH login failing on stable/13

On 2022-Jan-10, at 21:34, bob prohaska <fbsd@www.zefox.net> wrote:

> On Mon, Jan 10, 2022 at 12:34:57PM -0800, Mark Millard wrote:
>> On 2022-Jan-10, at 11:39, bob prohaska <fbsd@www.zefox.net> wrote:
>> 
>>> A Pi3 running stable/13 updated yesterday has stopped accepting ssh logins.
>>> The problem appeared after an update a couple days ago. The update was 
>>> repeated yesterday in hopes of catching some missing details, but no luck.
>>> 
>>> A session originated from RaspiOS with -vvv ends showing only 
>>> debug3: authmethod_lookup keyboard-interactive
>>> debug3: remaining preferred: password
>>> debug3: authmethod_is_enabled keyboard-interactive
>>> debug1: Next authentication method: keyboard-interactive
>>> debug2: userauth_kbdint
>>> debug3: send packet: type 50
>>> debug2: we sent a keyboard-interactive packet, wait for reply
>>> debug3: receive packet: type 60
>>> debug2: input_userauth_info_req
>>> debug2: input_userauth_info_req: num_prompts 1
>>> Password for bob@pelorus.zefox.org:
>>> debug3: send packet: type 61
>> 
>> 61 is, apparently, SSH_MSG_USERAUTH_INFO_RESPONSE
>> (No surprise.)
>> 
>>> Connection closed by 50.1.20.24 port 22
>>> 
>>> The "connection closed" took a couple of minutes to appear.
>>> Other ssh connections to older versions of FreeBSD seem to
>>> work normally. I looked at bugzilla, nothing recent about
>>> ssh. 
>> 
>> As a means of information gathering, when the RPi3 is running
>> and older FreeBSD, can you try the -vvv activity and report
>> the debug output, going a few packets past the type 61 notice
>> to see what would normally be next?
>> 
> Alas, no. I updated both of my Pi3's recently, and both seem
> to have similar problems. 

Why not put a recent snapshot or artifacts build on a microsd
card and try just that? Similarly for checking an older release,
from a time frame you know your build(s) for your context were
working?

No need to mess up your normal-use context. If only your
normal-use contexts fail, that would be interesting information
as well.

>> Also, do you have an alternative to using RaspiOS (or, even,
>> avoiding Linux) for such a test?: checking if it is somehow
>> specific to FreeBDS vs. RaspiOS/Linux or not? (Another FreeBSD
>> or NetBSD or . . . For FreeBSD, trying having both machines
>> using the problematical FreeBDS version could be interesting
>> if that is the only combination that works, for example.)
>> 
> Yes, I've tried a Pi2 running 12.3, which ends with
> debug2: we sent a keyboard-interactive packet, wait for reply
> debug3: receive packet: type 60
> debug2: input_userauth_info_req
> debug2: input_userauth_info_req: num_prompts 1
> Password for bob@pelorus.zefox.org:
> debug3: send packet: type 61
> Connection closed by 50.1.20.24 port 22
> 
> Also tried 14-current, same outcome.

Intersting.

>> Have you checked for any console messages, dmesg -a output,
>> or less /var/log/messages output that might be related?
>> (This likely would require serial console access.)
> 
> Yes, I have serial console access and there's been nothing
> on the console.
> 
> One oddity, however. If I send pings to the troublesome host
> one or two packets make it back with normal round trip time. 
> Then silence. Then a few more packets:

The way I read the below, icmp_seq=0 and icmp_seq=1 took
more time than icmp_seq=58 and icmp_seq=117 and the like.
But lots of icmp_seq values did not complete a round trip.

Also:

 58-1   == 57
176-118 == 58
235-177 == 58

It looks like the start of a pattern already.

> bob@nemesis:~ % ping pelorus.zefox.org
> PING pelorus.zefox.org (50.1.20.24): 56 data bytes
> 64 bytes from 50.1.20.24: icmp_seq=0 ttl=63 time=2.976 ms
> 64 bytes from 50.1.20.24: icmp_seq=1 ttl=63 time=2.073 ms
> 64 bytes from 50.1.20.24: icmp_seq=58 ttl=63 time=1.662 ms
> 64 bytes from 50.1.20.24: icmp_seq=117 ttl=63 time=1.519 ms
> 64 bytes from 50.1.20.24: icmp_seq=118 ttl=63 time=1.555 ms
> 64 bytes from 50.1.20.24: icmp_seq=176 ttl=63 time=1.570 ms
> 64 bytes from 50.1.20.24: icmp_seq=177 ttl=63 time=1.586 ms
> 64 bytes from 50.1.20.24: icmp_seq=235 ttl=63 time=1.528 ms 
> I'll let it run overnight, perhaps a pattern will emerge.

Intersting. I wonder what various statistics are like over
such activities on both sides of the communications. Dropped
packets? Other types of error counts?

(I do not claim to know much about what statistics are
available or the implications of what might be reported.)

> Bootup is normal, time setting is successful and outgoing
> connections seem normal. 
> 
> I'm trying to finish an update to the Pi3 running -current
> (via serial console) and will shortly see if that works any
> better.
> 




===
Mark Millard
marklmi at yahoo.com