FreeBSD-10.3-RELEASE-i386-memstick.img installer changes contents of USB flash drive

[Ian Smith]

On Thu, 29 Sep 2016 07:04:13 -0600, Warren Block wrote:
 > On Thu, 29 Sep 2016, Ian Smith wrote:
 > 
 > > In freebsd-questions Digest, Vol 643, Issue 3, Message: 17
 > > On Tue, 27 Sep 2016 14:19:39 -0700 David Christensen
 > > <dpchrist at holgerdanske.com> wrote:
 > > > On 09/27/2016 08:12 AM, Ian Smith wrote:
[..]
 > > > # cmp -l
 > > >
 > > /mnt/i3000d/data/dpchrist/iso/freebsd/10.3/FreeBSD-10.3-RELEASE-i386-memstick.img
 > > > /dev/sdc
 > > > 691669010 351 235		<< ~18 bytes into the last 512-byte block
 > > > 691669011 117 252
 > > > 691669012   2 351
  [.. lots ..]
 > > > 691669495   0 227
 > > > 691669497   0  22
 > > > 691669498   0 163
 > > > 691669499   0 362
 > > > 691669500   0 372
 > > > 691669501   0 221
 > > > 691669502   0 230
 > > > 691669503   0 122
 > > > 691669504   0 256
 > > > cmp: EOF on
 > > >
 > > /mnt/i3000d/data/dpchrist/iso/freebsd/10.3/FreeBSD-10.3-RELEASE-i386-memstick.img
 > > 
 > > Right, so most of the last sector was updated.  I'm no good at octal ..
 > > 
 > > We discerned recently that for 10.3, i386 memstick is GPT/EFI and amd64
 > > is still MBR/BSD.  Since only the last sector differs, perhaps something
 > > has updated the secondary GPT during the installation, assuming that a
 > > GPT image should already include one - or at least, the space for one?
 > 
 > A secondary copy of the partition table is saved at the end of a 
 > GPT-partitioned drive. This is a problem for GPT when a binary image is
 > written to a device that is not exactly the same size as the image. That is
 > pretty much always the case with installer images.

Indeed.  MBR/BSD images don't have this problem because all metadata is 
at the beginning of and within the image in the case of UFS superblocks.  
So an image can be copied to a device like /dev/da0, in the case of the 
hitherto dedicated drive /dev/da0a, or as easily to a slice like da0s1 
where it becomes da0s1a, as long as there's enough room on the memstick.

A GPT image seems more of a whole-disk image rather than (what equally 
can serve as) a single partition image.  It must have a primary GPT (all 
34 sectors?) but do you know if these images already contain the full 34 
sector secondary GPT?  i.e, I don't see why only the last sector of the
copied image file was changed, with no update to any of the prior 33?

 > gpart can repair
 > ("recover") the backup partition table because it is a copy of the primary
 > partition table.  This might be required during an install to make sure the
 > disk works with particularly strict BIOS or UEFI implementations.  Although
 > it seems like a reboot would be required afterwards.

Hmm, well it's already booted from the memstick and installed the OS, so 
we know the installer disk works in that environment.  I can't see any 
need to be modifying the source disk when it's doing / done its job?

And then Perry Hutchinson (cc'd) is having the 'opposite' problem of 
wanting to update the 10.3 i386 memstick he'd booted off with gpart 
recover and gpart add for another partition, apparently successfully 
according to gpart show, only to have the updates disappear on reboot?

Personally I'm wanting to get to a memstick with 2 or 3 bootable slices 
for installers (amd64 and i386) - as least the amd64 one from dvd1 with 
packages - plus a mountable r/w data partition.  Using only native tools 
currently implies using boot0, thus MBR/BSD schema.  Converting a new 
GPT-scheme memstick image to ONE slice on my new 16GB memstick is what I 
need to figure out, possibly by means of using a tar pipeline similar to 
dvd1_to_memstick.sh.  I hope to get time to look at 11.0 make release 
memstick scripts, but I'd appreciate any clues / ideas meanwhile ..

cheers, Ian  (who didn't mention f@!$k once! :)