Upgrading an i386 machine from amd64.

[Alfred Perlstein]

Hello hackers.

Is there a way to build on amd64 and then mount over nfs the build and 
src and installworld from an i386 machine?

The problem seems to be that "install" and "strip" and etc are built as 
amd64 binaries so that the installworld will fail.

Below I have a solution I was going to do a blog post about, but then 
realized maybe I'd be leading people down the wrong path.

Can someone verify that I need to use rsync as opposed to installworld 
for this to work?

I have an old i386 based soekris geode box called "soekris":
CPU: Geode(TM) Integrated Processor by AMD PCS (499.91-MHz 586-class CPU)
   Origin = "AuthenticAMD"  Id = 0x5a2  Family = 0x5  Model = 0xa 
Stepping = 2
Features=0x88a93d<FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CLFLUSH,MMX>
   AMD Features=0xc0400000<MMX+,3DNow!+,3DNow!>
real memory  = 536870912 (512 MB)
avail memory = 502792192 (479 MB)

Building on this machine is difficult because of the speed and lack of 
space, so I decided to use my more powerful amd64 machine "spigot":
CPU: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz (3392.37-MHz K8-class CPU)
   Origin="GenuineIntel"  Id=0x206a7  Family=0x6  Model=0x2a Stepping=7
...
real memory  = 17179869184 (16384 MB)
avail memory = 16585228288 (15816 MB)
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 SMT threads

So I've built an 10-stable on the amd64 machine using:

spigot % TARGET=i386
spigot % make -j24 buildworld
spigot % make -j24 buildkernel

Works great, I get an i386 object tree under /usr/obj/i386.386/...

Then I go to install this over NFS and this is where I get stuck.

If I mount the i386 machine like so and install I get errors on libc and 
other libraries:

spigot % mount soekris:/ /usr/soekris
spigot % cd /usr/src && make installworld
.....
===> lib/libcrypt (install)
install -C -o root -g wheel -m 444   libcrypt.a /usr/soekris/usr/lib
install -C -o root -g wheel -m 444   libcrypt_p.a /usr/soekris/usr/lib
install -s -o root -g wheel -m 444   -fschg  libcrypt.so.5 /usr/soekris/lib
install: /usr/soekris/lib/libcrypt.so.5: Input/output error
*** Error code 71

Stop.
make[5]: stopped in /usr/trees/freebsd.git/lib/libcrypt
*** Error code 1

OK, so that doesn't work...

Maybe if I mount the amd64 build host under the soekris box, no that 
breaks because the bootstrap tools (install(1), strip(1)) are built for 
amd64 so the install fails.

So what I finally did on the amd64 box was:

spigot % mkdir /use/soekris.local
spigot % make installworld DESTDIR=/usr/soekris.local
spigot % mount soekris:/  /usr/soekris
spigot % rsync -avvH /usr/soekris.local/ /usr/soekris/

That hung at the end forever at the end:
var/unbound/
var/yp/
var/yp/Makefile
var/yp/Makefile.dist
....

but after giving it an hour I just hit ^C and rebooted and everything 
was more or less fine.

Is there a better way to do this?  Was the "installworld to NFS" 
breaking because of NFS bugs?  Should I get those to Rick?  I'm just 
confused.

-Alfred