GEOM vinum and SMP (AMD64)

[Nyaano]

Ulf Lilleengen wrote:
> On Sun, Oct 15, 2006 at 10:30:54PM +0200, Nyaano wrote:
>   
>> Hi,
>>
>> I've just tried out FreeBSD 6.2-BETA2 (AMD64) on a Core2Duo E6300.
>> I installed the SMP kernel without any problem, made a gvinum 
>> configuration and created a RAID5 volume.
>> I can mount it without any problems, and I added geom_vinum_load="YES" 
>> in /boot/loader.conf and the appropriate line in /etc/fstab after 
>> creating a filesystem on the volume.
>> After that, I do a simple shutdown -r now because I want to make sure 
>> that everything starts up the way it should.
>> When the system comes to the point where it's going to mount the RAID5 
>> volume, it complains about the superblock and says it's corrupt.
>>
>> With exactly the same configuration, but with the GENERIC kernel, 
>> everything works without any problems.
>>
>> Before I spend more time on trying to get this working with SMP, I'd 
>> like to know if this is a known problem, and if it's not, what 
>> information is needed from me in order to track down what the problem is.
>>     
>
> Hi,
>
> Could you try to boot into single-user mode, try fsck on the volume, and then
> mount it? It would also be nice if you could send your config as well as any
> relevant error message.
>
>   

The computer in question had the following setup (I write had because I 
had to put it to other use for now, will make a new attempt with FreeBSD 
when I can free it up again):
Asus P5B
Core 2 Duo E6300
1GB RAM
4x320GB WD3200KS on the mobo ICH8-controller (configured in BIOS as IDE, 
not AHCI because with AHCI FreeBSD would only detect two drives)
2x250GB WD2500KS on a Highpoint 2300 SATA-controller (using driver 
provided by Highpoint, configured for RAID1)
1 SATA CD/DVD on the JMicron controller (CD/DVD not detected by FreeBSD 
in either IDE or AHCI mode in BIOS)
Some noname firewire PCI card
Some nVidia PCI-Express graphics card


What I did was install a minimal FreeBSD 6.2-BETA2 with SMP kernel from 
AMD64 iso, proceeded to run gvinum and produced a configuration that put 
the 4 320GB drives in a RAID5 volume, created filesystem with newfs -U, 
added lines in /boot/loader.conf (geom_vinum_load="YES") and /etc/fstab 
(can't recall exactly, but it was a standard line).

After shutdown -r now it booted up fine but when it came to mounting the 
volume, it would complain about corrupt superblock (sorry, don't have 
exact error message). I tried to run fsck on the /dev/gvinum/fileserver 
volume, but it would complain about the superblock.

To see if the same thing happened with GENERIC kernel, I installed from 
scratch and picked GENERIC instead, started gvinum and let that load the 
RAID5 config from the disks, then put on a new filesystem. shutdown -r 
now and it booted up to the prompt without any problems.

Before I had to put the computer to other use, I tried again with SMP, 
and this time everything worked fine. I simply didn't have any more time 
to test these things further, but I started to question my own 
assumption that the problem was in GEOM vinum, as that should be well 
tested in both GENERIC and SMP configs, and started to wonder if the 
problem could be in the ICH8 driver instead. Lack of time, and not being 
too familiar with FreeBSD forced me to abandon this for now though. (I 
had previously already lost a lot of time from wondering how the network 
support could be so flaky during install until I discovered that it was 
the cable that was at fault and not the network card or FreeBSD, as well 
as trying to get FreeBSD to notice my SATA CD/DVD). I should be able to 
make a new attempt at this in a few weeks which should be a lot less 
painful experience :p

Regards,
Nyaano