freebsd-smp Digest, Vol 101, Issue 4

[M. J. [Mike] O'Brien]

On Sat, July 16, 2005 8:00 am, freebsd-smp-request at freebsd.org said:
> Send freebsd-smp mailing list submissions to
> 	freebsd-smp at freebsd.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://lists.freebsd.org/mailman/listinfo/freebsd-smp
> or, via email, send a message with subject or body 'help' to
> 	freebsd-smp-request at freebsd.org
>
> You can reach the person managing the list at
> 	freebsd-smp-owner at freebsd.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of freebsd-smp digest..."
>
>
> Today's Topics:
>
>    1. FreeBSD unstable on Dell 1750 using SMP? (Rutger Bevaart)
>    2. Re: FreeBSD unstable on Dell 1750 using SMP? (Lucas Holt)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 15 Jul 2005 15:15:24 +0200 (CEST)
> From: "Rutger Bevaart" <rutger.bevaart at illian.net>
> Subject: FreeBSD unstable on Dell 1750 using SMP?
> To: freebsd-smp at freebsd.org
> Message-ID: <24434.193.172.18.3.1121433324.squirrel at 193.172.18.3>
> Content-Type: text/plain;charset=iso-8859-1
>
>
> hello list,
>
> For the past year we've been running several Dell PowerEdge 1750 servers
> on FreeBSD 4.10, 4.11 and 5.3. All these machines have dual Xeons running
> with HT enabled. This install has proven to be unstable in that the
> machine will reboot between 3 days and 170 days without apparant reason.
> No log is written. Other machines we have with a single CPU (HT enabled)
> do not experience this problem.
>
> As it is present in both 4.x and 5.x and googling the last year has not
> revealed similar experience I'm consulting this list. As all of these
> machines are productions machines that have a continuous load (not heavly
> load, but a light average - some peaks) it's not easy to experiment with
> HT setting etc. I dislike driving to the datacenter for locked systems
> with fubarred kernels ;-)
>
> The only error i've ever seen just before a reboot is "bge0: discard frame
> w/o packet header" on the 5.3 machine.
>
> Any clues or help greatly appreciated!
>
> Regards
> Rutger Bevaart
>
>
>
> ------------------------------
>
> Message: 2
> Date: Fri, 15 Jul 2005 21:47:46 -0400
> From: Lucas Holt <luke at foolishgames.com>
> Subject: Re: FreeBSD unstable on Dell 1750 using SMP?
> To: Rutger Bevaart <rutger.bevaart at illian.net>
> Cc: freebsd-smp at freebsd.org
> Message-ID: <3713FA02-FDBB-4B24-A592-F55B7A485C26 at foolishgames.com>
> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
>
> I can't speak for that config or network card, but I had a similar
> experience with freebsd 5.2 and 5.3.  I got unusual errors
> occasionally for an rl nic and the machine randomly rebooted.
> Sometimes nothing was logged.  It turned out to be the network card.
> I replaced the NIC with a 3com 3c905c TX and the problem went away.
>
> Its possible that the dell NICs are non standard and the driver isn't
> handling them well.  I've noticed problems with dell nics and
> standard drivers in their other products (lattitude d800, etc).  If
> you really thought it was an SMP issue, I suppose you could compile
> and run a non smp kernel as a test.
>
> Its also possible that "4" processor smp isn't as reliable as 2.  I
> have read about scalability issues in the past with large numbers of
> cpus in freebsd.
>
> On Jul 15, 2005, at 9:15 AM, Rutger Bevaart wrote:
>
>>
>> hello list,
>>
>> For the past year we've been running several Dell PowerEdge 1750
>> servers
>> on FreeBSD 4.10, 4.11 and 5.3. All these machines have dual Xeons
>> running
>> with HT enabled. This install has proven to be unstable in that the
>> machine will reboot between 3 days and 170 days without apparant
>> reason.
>> No log is written. Other machines we have with a single CPU (HT
>> enabled)
>> do not experience this problem.
>>
>> As it is present in both 4.x and 5.x and googling the last year has
>> not
>> revealed similar experience I'm consulting this list. As all of these
>> machines are productions machines that have a continuous load (not
>> heavly
>> load, but a light average - some peaks) it's not easy to experiment
>> with
>> HT setting etc. I dislike driving to the datacenter for locked systems
>> with fubarred kernels ;-)
>>
>> The only error i've ever seen just before a reboot is "bge0:
>> discard frame
>> w/o packet header" on the 5.3 machine.
>>
>> Any clues or help greatly appreciated!
>>
>> Regards
>> Rutger Bevaart
>>
>> _______________________________________________
>> freebsd-smp at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-smp
>> To unsubscribe, send any mail to "freebsd-smp-unsubscribe at freebsd.org"
>>
>
>
> Lucas Holt
> Luke at FoolishGames.com
> ________________________________________________________
> FoolishGames.com  (Jewel Fan Site)
> JustJournal.com (Free blogging)
> FoolishGames.net (Enemy Territory IoM site)
>
> Think PC.. in 2006 you can own an Apple PCintosh. Whats next, windows
> works?
>
>
>
> ------------------------------
>
> _______________________________________________
> freebsd-smp at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-smp
> To unsubscribe, send any mail to "freebsd-smp-unsubscribe at freebsd.org"
>
> End of freebsd-smp Digest, Vol 101, Issue 4
> *******************************************
>
Suggest: Disable HT...
I had the same problem of unexplained server restarts on 5.3 last December
and strongly suspected a DMA issue or a dual channel memory paging foobar
when HT was enabled. The benefits of HT over simply running SMP on just
two physical CPUs are apparently zero in your case as loads seem light and
therefore spreading threads across more CPUs does not seem to be an
advantage. In short, turn hyperthreading off. In my case that's what I did
and the crashes (which also seemed to be about three days apart or so)
stopped.

Good luck.

best... Mike