Gmirror - need information and help

[Conrad Burger]

Hi all 

Hope someone can help me with the following problem(s).

The scenario: 2 system and 5 disks. Freebsd 5.3-p5 

I created a 3 member gmirror mirror on system1. 
Removed one member(disk) from system1. (Shut the system and removed disk)
Then inserted the disk in system2 and booted the system.

Now the problem is: on system1 I have a "2 member gmirror" and on system2 I have
a "1 member gmirror. And I can't add more members to the gmirror on system2.

System1's gmirror is in a degraded state. How do I "tell" gmirror to remove the
"removed disk" from its configuration and change its state to COMPLETE?

What do I need to do to be able to add a member to System2's gmirror?

Below are the output of how I tried to add another member to system2's gmirror
and the output of "gmirror list" on both systems.

Please help, I have been struggling with this problem for the last week. The
systems need to go into production!

Will provide more info per request...

Regards
Conrad 



Output of system2 (trying to add another member to gmirror(gm0))
----------------------------------------------------------------------
[63] root at boron:/var/log# glabel list
Geom name: da0
Providers:
1. Name: label/disk2
   Mediasize: 18373348864 (17G)
   Sectorsize: 512
   Mode: r5w5e3
   secoffset: 0
   offset: 0
   seclength: 35885447
   length: 18373348864
   index: 0
Consumers:
1. Name: da0
   Mediasize: 18373349376 (17G)
   Sectorsize: 512
   Mode: r5w5e4

[65] root at boron:/var/log# glabel label -v disk0 da1
Metadata value stored on da1.
Done.

[66] root at boron:/var/log# glabel list
Geom name: da1
Providers:
1. Name: label/disk0
   Mediasize: 18373348864 (17G)
   Sectorsize: 512
   Mode: r0w0e0
   secoffset: 0
   offset: 0
   seclength: 35885447
   length: 18373348864
   index: 0
Consumers:
1. Name: da1
   Mediasize: 18373349376 (17G)
   Sectorsize: 512
   Mode: r0w0e0

[70] root at boron:/var/log# gmirror insert gm0 label/disk0
Not all disks connected.


----------------------------------------------------------------------

Output of system1 (gmirror list)
----------------------------------------------------------------------
Geom name: gm0
State: DEGRADED
Components: 3
Balance: load
Slice: 4096
Flags: NONE
SyncID: 3
ID: 2033501248
Providers:
1. Name: mirror/gm0
   Mediasize: 18373348352 (17G)
   Sectorsize: 512
   Mode: r5w5e2
Consumers:
1. Name: label/disk0
   Mediasize: 18373348864 (17G)
   Sectorsize: 512
   Mode: r5w5e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 3
   ID: 407245497
2. Name: label/disk1
   Mediasize: 18373348864 (17G)
   Sectorsize: 512
   Mode: r5w5e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 3
   ID: 1435311995

Geom name: gm0.sync
----------------------------------------------------------------------

Output of system2 (gmirror list)
----------------------------------------------------------------------
Geom name: gm0
State: DEGRADED
Components: 3
Balance: load
Slice: 4096
Flags: NONE
SyncID: 5
ID: 2033501248
Providers:
1. Name: mirror/gm0
   Mediasize: 18373348352 (17G)
   Sectorsize: 512
   Mode: r5w5e2
Consumers:
1. Name: label/disk2
   Mediasize: 18373348864 (17G)
   Sectorsize: 512
   Mode: r5w5e3
   State: ACTIVE
   Priority: 0
   Flags: DIRTY
   SyncID: 5
   ID: 1488378826

Geom name: gm0.sync
----------------------------------------------------------------------