FBSD 10.3 + ZFS + Sun x4500 = utter lock up.

[Lee Damon]

On 2/8/17 05:53 , Ronald Klop wrote:
> ...  
> Any reason not to try 11? I don't know if it fixes anything, but it
> would be a nice data point for comparison.

I'll probably give that a try shortly but given the problem was seen in
previous releases I'm not optimistic. I'm only in that location on
Monday and Tuesday so can't try anything until next week.

> Ronald.
> 
> 
> On Tue, 07 Feb 2017 22:44:01 +0100, Lee Damon <nomad at castle.org> wrote:
> 
>> FreeBSD [redacted] 10.3-STABLE FreeBSD 10.3-STABLE #2 r313008: Tue Jan
>> 31 01:50:49 PST 2017     lvd@[redacted]:/usr/obj/usr/src/sys/GENERIC 
>> amd64
>>
>> I'm trying to get FBSD 10.3 working on a Sun x4500 (don't ask) for use
>> as a ZFS-based backup server. However, whenever any amount of data is
>> put into a zpool and then zpool scrub is run the host locks up hard. On
>> reboot it complains that a "Hyper transport sync flood occurred".
>>
>> I found
>> https://lists.freebsd.org/pipermail/freebsd-stable/2012-January/065542.html
>>
>> which seems to match but when I try the cpuset command mentioned there I
>> get an error:
>>
>> ; sudo cpuset -c -l 0 -x 58
>> cpuset: setaffinity: Invalid argument
>>
>> Looks like the -c was invalid. After removing that I was informed -x 58
>> wasn't valid. Sure enough, there's no mpt0 or IRQ 58 on the host:
>>
>> ; vmstat -i
>> interrupt                          total       rate
>> irq17: ohci2                        8578          2
>> irq18: ohci3                         473          0
>> irq19: ohci0 ohci1+                 4924          1
>> irq24: mvs0                          457          0
>> irq32: mvs1                          453          0
>> irq38: mvs2                          451          0
>> irq46: mvs3                         8063          1
>> irq52: em0                        152354         35
>> irq53: em1                           140          0
>> irq68: mvs4                          450          0
>> irq76: mvs5                          454          0
>> cpu0:timer                        208311         48
>> cpu1:timer                         98318         23
>> cpu2:timer                        105704         24
>> cpu3:timer                        106202         24
>> Total                             695332        162
>>
>> Looking around with some help from #freebsd on efnet I found mvs0-5
>> which are connected to the Marvel drive controllers on the host. I then
>> used
>>   ; sudo cpuset -l 0 -x ##
>> where I replaced ## with 24, 32, 38, 46, 68, and 76.
>>
>> After rebuilding the zpool I started writing to it. It took a lot less
>> time to crash - I didn't even need to run zpool scrub - but instead of
>> completely locking up it just rebooted. I did not see reference to the
>> hyper transport problem while watching it boot but given the poor
>> performance of the serial console I can't be 100% sure it wasn't there.
>>
>> So now I turn here to ask for guidance. Is anyone currently successfully
>> running 10.x on a x4500 and if so, how are you doing it? If not, how can
>> I get this working?
>>
>> thanks,
>> nomad
>> _______________________________________________
>> freebsd-stable at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"