Gmirror + AXP + 5.4.. (was Vinum + AXP + 5.4)

[Bernd Walter]

On Fri, Jul 29, 2005 at 03:28:31PM +0100, Angus MacGyver wrote:
> 
> Pawel Jakub Dawidek said:
> > On Wed, Jul 27, 2005 at 09:46:48AM +0100, Angus MacGyver wrote:
> > +> On Wed, Jul 27, 2005 at 09:05:37AM +0200, Pawel Jakub Dawidek wrote:
> > +> > On Wed, Jul 27, 2005 at 07:47:29AM +0100, Angus MacGyver wrote:
> > +> > +> My fstab only has refereence to / as /dev/mirror/md0 and nothing
> > else.
> > +> > +>
> > +> > +> /dev/mirror/md0         /               ufs     rw      1   1
> > +> >
> > +> > But could you mount /dev/mirror/md0 on /mnt and confirm that
> > +> > / is /dev/mirror/md0 in /mnt/etc/fstab?
> > +>
> > +>
> > +> and one hears the sound of a slap on the forehead......
> > +> DOH!
> > +>
> > +> I'd newfs'd /dev/mirror/md0 and then dumped / to this, BEFORE i'd
> > changed the fstab to point to /dev/mirror/md0....
> > +>
> > +> When the machine booted and picked up the fstab from /dev/da0a, it was
> > pointing / at /dev/da1a, from which is read /etc/fstab for all the other
> > mounts, which then mounted as expected....
> > +>
> > +>
> > +> Talk about muppetry...
> > +>
> > +> I now have...
> > +>
> > +>
> > +> /dev/mirror/md0 on / (ufs, local, soft-updates)
> > +> devfs on /dev (devfs, local)
> > +> /dev/mirror/md2 on /var (ufs, local, soft-updates)
> > +> /dev/mirror/md3 on /tmp (ufs, local, soft-updates)
> > +> /dev/mirror/md4 on /usr (ufs, local, soft-updates)
> > +> /dev/mirror/md5 on /data (ufs, local, soft-updates)
> > +> procfs on /proc (procfs, local)
> > +>
> > +> Precisely what I'd expected...
> > +>
> > +> Thanks for all your help ;-)
> >
> > No problem, good to hear to works fine:)
> 
> 'Cept i have now has UFS panics...
> 
> Upon creating a jail using Ths Lab's webmin insert, and creating a disk
> based install, of which there are already 3 created and installed..
> 
> /data/test: bad dir ino 60 at offset 0: mangled entry
> panic: ufs_dirbad: bad dir
> Uptime: 3h21m13s
> 
> ... and so starts a reboot..
> 
> NOW..
> 
> Question i have to ask is this, as I cannot find a clear statement, and
> "man gmirror" only appears to hint at the metadata...
> 
> I installed using sysinstall, and split the disk thus..
> 
> corse# bsdlabel /dev/da0
> # /dev/da0:
> 8 partitions:
> #        size   offset    fstype   [fsize bsize bps/cpg]
>   a:   524288        0    4.2BSD     2048 16384 32776
>   b:  2099200   524288    4.2BSD     2048 16384 28528
>   c: 35843670        0    unused        0     0         # "raw" part,
> don't edit
>   d:  1048576  2623488    4.2BSD     2048 16384     8
>   e:  1048576  3672064    4.2BSD     2048 16384     8
>   f:  7340032  4720640    4.2BSD     2048 16384 28528
>   g: 23782998 12060672    4.2BSD     2048 16384 28528
> 
> 
> I then copied this label to the second drive and setup the mirrors as
> described earlier.
> (for those that didnt see it....)
> corsec# gmirror status
>       Name    Status  Components
> mirror/md0  COMPLETE  da0a
>                       da1a
> mirror/md1  COMPLETE  da0b
>                       da1b
> mirror/md2  COMPLETE  da0d
>                       da1d
> mirror/md3  COMPLETE  da0e
>                       da1e
> mirror/md4  COMPLETE  da0f
>                       da1f
> mirror/md5  COMPLETE  da0g
>                       da1g
> 
> i.e. for each side of the mirror, I am using an identically sized slice
> (partition/lump) of each disk..
> 
> Now, question is, where does it store the metadata ?
> Manual says last sector.
> Ok, so in my case, is that last sector of c? (the whole disk), for ALL
> mirror devices, or the last sector of each slice on each disk ???

The last sector of the provider, e.g. da0a/da1a for md0 in your case.
Normally one would use the entire disks and then bsdlabel the mirror.

> So what happens when disks ineveitably get full, is the metadata "safe" ?

Normaly the resulting disk should be a sector smaller than the
provider, but if that really happened that wouldn't explain this error.
Typically this kind of problems arise if a mirror is our of sync.
I would suggest a resync of the mirror and then a forced fsck.

> Or would I be best off starting again, and creating a mirror device that
> has the ENTIRE disk as the device, and then slice that mirror device up
> for /, /var /usr etc...

That's what most people do.

> I am used to Solaris Disk Suite's mirroring, and we setup a slice on each
> disk JUST to keep the metadata in, so this causing me some brain troubles
> at the moment..

Special partitions for metadata  is nice, but Disk Suite fails
miserably if you shuffle disks or have to move a raid to another
machine, because it identifies disks by devicenames, while GEOM
identifies eachs provider by it's metadata.

> Most of the "howto" doc are x86 and only for IDE devices, neither of which
> I am using.

Beside that RAID on IDE scares me there is not much difference, just
that we don't use fdisk-style slices on alpha.

-- 
B.Walter                   BWCT                http://www.bwct.de
bernd at bwct.de                                  info at bwct.de