Sudden Reboots

[Jim Durham]

On Monday 04 October 2004 04:04 pm, Jim Durham wrote:
> On Monday 04 October 2004 03:06 pm, Doug Russell wrote:
> > On Mon, 4 Oct 2004, Jim Durham wrote:
> > > The reboots started out happening at 5.15 pm or so. I had them unplug
> > > the server completely from AC and restart it and now it's happening
> > > withing a few minutes of 12:40pm every day.
> > >
> > > The 'last' command output is the only thing showing anything log-wise.
> > > Look at this:
> > >
> > > reboot           ~                         Mon Oct  4 12:33
> > > reboot           ~                         Sun Oct  3 12:37
> > > reboot           ~                         Sat Oct  2 12:42
> > > reboot           ~                         Fri Oct  1 12:45
> >
> > That is strange....
> > What is that machine doing at that time?  Does it start a backup job, or
> > any kind of maintenance at that time?  Anything at all?
> >
> > I'd monitor what the heck the machine is doing to some remote machine and
> > see what goes on when it dies.
> >
> > Later.......      <Doug>
>
> Well... I'm trying to do too many things at the same time and not thinking.
> I thought about it and also ran 'last -f /var/log/wmtp.1' and found the
> whole sequence...and added my recent reboot info to the top and here is
> what I got.
>
>
> reboot           ~                         Mon Oct  4 14:54
> reboot           ~                         Mon Oct  4 12:33
> reboot           ~                         Sun Oct  3 12:37
> reboot           ~                         Sat Oct  2 12:42
> reboot           ~                         Fri Oct  1 12:45
> reboot           ~                         Thu Sep 30 12:33
> reboot           ~                         Thu Sep 30 08:54
> reboot           ~                         Wed Sep 29 08:41
> reboot           ~                         Tue Sep 28 17:20
> reboot           ~                         Mon Sep 27 17:28
> reboot           ~                         Sun Sep 26 17:28
> reboot           ~                         Sat Sep 25 17:37
> reboot           ~                         Fri Sep 24 17:38
> reboot           ~                         Thu Sep 23 17:42
> reboot           ~                         Wed Sep 22 17:50
> reboot           ~                         Tue Sep 21 17:52
> reboot           ~                         Mon Sep 20 17:57
> reboot           ~                         Sun Sep 19 17:57
> reboot           ~                         Sat Sep 18 17:56
>
> Now, the machine was powered down on Sept 17 because of an all-day power
> failure. It started doing this on the 18th ,starting at 5:56pm and working
> earlier until Sept 29, when I upgraded from 4.9 to 4.10 p3. Then it started
> rebooting at 8:45 aprox and I had the customer shut down the machine
> completely on the 30th and remove the AC from the rear connector and wait a
> few minutes and bring it back up. That's when it started rebooting at 12:30
> or so.
>
> Googling for "Freebsd  reboots at same time every day"  will produce hits
> on people with the same problem so this is not just me.
>
> I checked crontab, etc and there's nothing coinciding and also, apparently
> every time you reboot the machine, the cycle changes, so how can it be an
> external event? Boy...I'm confused...  8-)
>
> There was one Google posting that maintained that this one guy fixed it by
> removing IPV6 from the kernel. I'm trying this at the moment. Of course, I
> had to reboot to change the kernel, so the times will probably change if it
> keeps rebooting.
>
> It also just hit me that the time doesn't change much if the machine
> reboots itself, but changes if you upgrade or do a controlled reboot. 
> Wow...

This is a followup posting on the above problem..

After removing IPV6 from the kernel and recompiling ,(note that I had 
recompiled the kernel earlier during this troubleshooting series with no 
effect) , the problem has gone away. The machine has been up for over 2 days 
with no reboots.

So, to recap:

Machine reboots at aprox 23hrs 57 minutes of uptime at ~12:30pm..
Upgraded to latest 4.10p3 sources.
Still reboots at 23hrs 57mins, but at different time of day, ~8:15am.
Unplugged machine totally from AC and restarted.
Still reboots at 23:57 intervals, but time is now changed again, ~2:55pm.
Recompiled kernel, removing IPV6.
Problem stops. Up now for over 2 days for first time in weeks.

Note1: The 23:57 is the usual interval. Sometimes it varied by a few minutes.
Note2:  'healthd' showed normal operation of Power Supply with no log entries.
Note3: The IPV6 idea was gleaned from a Google search of earlier postings on 
same problem.
Note4: Another posting claimed a cure of this problem by changing network 
card.

Observation: When the machine reboots itself, the time of the reboots seems to 
stay very close to 23:57 intervals. When you reboot on command, the interval 
stays the same, but the time of day changes, and does not coincide with the 
commanded reboot time. I have no conclusion about this.

Thanks to all who made suggestions.

-Jim Durham