[SunHELP] Cloning disks on SunBlade 1000's and Serial Numbers.

[Ray Arachelian]

Hey guys,

I'm not actually asking for help, but rather offering it incase anyone
else runs into the same issues.  If someone's maintaining a FAQ, this
would be a good thing to add to it.


One of my recent fun tasks was to build a whole slew of SunBlade 1000's.
This sounded like a job for either JumpStart or good ol'e dd.  I opted for
dd of course as the lesser of the two evils.   Why?  It's a pain to set up
JumpStart to begin with, it eats up a lot of bandwith when you have lots
of machines to setup, and it's much slower than cloning a whole drive.

And if anything goes wrong, you'll have to mess around with the configuration
all over again, jump all over again, and repeat until you get it to work.
Too messy for my taste.  Instead, I built one box with everything I wanted
exactly as I wanted it, and use dd/installboot.

But then I ran into something wicked.

I
and c1t2d0 is the top drive - the disk I'm writing to.

i.e.

  dd if=/dev/dsk/c1t1d0s2 of=/dev/dsk/c1t2d0s2 bs=65536
and then all you have to do is install the bootblock like this:

/usr/sbin/installboot /usr/platform/SUNW,Sun-Blade-1000/lib/fs/ufs/bootblk
/dev/rdsk/c1t2d0s0


Now, when I tried to boot off the cloned disk, the kernel panics with
error messages stating it couldn't find /var or /usr.

I know what you're about to say: SCSI defect list lives in the first cylinder
and I've probably blown it away, yes?  Well, no that wasn't it at all.  I
later
tried formatting the disk with identically sized partitions, and tried to dd
slice by slice to avoid that.  It also wasn't the block size passwd to dd.  I
even used the usual 512 (really slow) block size to no avail.

I even tried newfs'ing the partitions and using ufsdump 0f - | ufsrestore -rf
to no avail.


What's happening is that Sol8 or the qlc controller is returning the serial 
number of the disk as a device ID instead of the SCSI id.  When you clone 
the disk, the copy will have a different serial number of course, but since
the old device paths pointed to the serial number of the original, you won't
be able to boot the OS.

I've confirmed this because I did find the fix, I was able to take the
now bootable cloned disk and pop it into a different machine, and boot.

The fix as expected is to run the disks and the drvconfig command.  But
before you say "Well Duh, that's obvious" let me point out that it did
not work!  Until I did something else.

I had to boot off the CD, mount the cloned disks' / and /usr to /mnt
and /mnt/usr, then run "chroot /mnt /bin/sh" and then run the following:

  disks; drvconfig; devfsadm -C -v; devfsadm -v; devfsadm -v -r /devices

Now, likely I didn't have to run all of that, but see in Solaris 8, the
disks command has been replaced with devfsadm, (see the man page) which
claims to be the preferred way of doing things.  The -v's you can skip,
that's just to be verbose.  I suspect you only need:

  disks; devfsadm -C; and devfsadm -r /devices

But once I got one of these puppies to work, I no longer cared to
waste any more time experimenting.

After that I did touch /reconfigure so when it boots it would
resolve any other device tree problems.

Then of course, you'd hit Control-D to get out of the chrooted shell,
and then unmount /mnt/usr and /mnt.  For good measure, I also
ran fsck -y /dev/dsk/c1t1d0s0 all the way to s7

Now, I did just ONCE manage to get one cloned disk to work without
having to chroot, but wasn't able to replicate that, and I likely did
a ton of other things such as tarring up /dev and /devices off the
CD and restore it to the clone drive.  All ugly unpleasant stuff that
mostly didn't work.  The chroot method worked every time.


Background info:

Solaris has never really used /dev as a place to keep block and character
devices.  Rather /dev contains soft links to /devices where the device
tree is kept.

On the SunBlade however, the qlc device tree as you can see contains the
serial number of the disks.

I.E:

Here's one SunBlade:

$ ls -Fla /dev/dsk/c1t1d0s0
lrwxrwxrwx   1 root     root          70 Mar 30 16:15 /dev/dsk/c1t1d0s0 -> 
../../devices/pci at 8,600000/SUNW,qlc at 4/fp at 0,0/ssd at w21000004cf72b741,0:a

Here's another:
# ls -Fla /dev/dsk/c1t1d0s0
lrwxrwxrwx   1 root     root          70 Apr  2 17:52 /dev/dsk/c1t1d0s0 -> 
../../devices/pci at 8,600000/SUNW,qlc at 4/fp at 0,0/ssd at w21000004cf728f45,0:a

And yet another:

lrwxrwxrwx   1 root     root          70 Apr  2 13:42 /dev/dsk/c1t1d0s0 -> 
../../devices/pci at 8,600000/SUNW,qlc at 4/fp at 0,0/ssd at w21000004cf72a421,0:a


As you can see the device path is different on every disk.  Yet I'm able
to take the disk out of one machine and boot it off another without a
problem (so long as the OBP is set up, etc...)



Here's my cloning script incase you didn't know how to clone disks under
solaris.  Just replace c1t1d0 with your source disk, and c1t2d0 with your
target disk.  I have slice 4 as swap.  

I placed this script on my source drive.  To use it, I boot off the CDROM
with -s, mount -o ro /dev/dsk/c1t1d0s0 /mnt; copy the script and other 
associated files to /tmp, umount /mnt, and run the script.  When the machine
powers off, it's done, and I remove the source disk, move the target disk
into the lower slot, and do the chroot repair as I stated above, then
boot off it.


I have a text file that I feed to the format command called format.txt

If this is of use to you, by all means, enjoy. :)  You'll probably need
to change the bootblock install path to whatever is appropriate for your
platform.  Replacing SUNW,Sun-Blade-1000 with `uname -i` would probably
do the trick.



#!/bin/sh

echo Checking Source disk FS
fsck -y /dev/dsk/c1t1d0s0
fsck -y /dev/dsk/c1t1d0s1
fsck -y /dev/dsk/c1t1d0s3
fsck -y /dev/dsk/c1t1d0s5
fsck -y /dev/dsk/c1t1d0s6
fsck -y /dev/dsk/c1t1d0s7

echo partitioning
format <format.txt

echo duplicating disk slices
dd if=/dev/dsk/c1t1d0s0 of=/dev/dsk/c1t2d0s0 bs=65536
dd if=/dev/dsk/c1t1d0s1 of=/dev/dsk/c1t2d0s1 bs=65536
dd if=/dev/dsk/c1t1d0s3 of=/dev/dsk/c1t2d0s3 bs=65536
dd if=/dev/dsk/c1t1d0s5 of=/dev/dsk/c1t2d0s5 bs=65536
dd if=/dev/dsk/c1t1d0s6 of=/dev/dsk/c1t2d0s6 bs=65536
dd if=/dev/dsk/c1t1d0s7 of=/dev/dsk/c1t2d0s7 bs=65536

echo Installing Bootblock
/usr/sbin/installboot /usr/platform/SUNW,Sun-Blade-1000/lib/fs/ufs/bootblk
/dev/rdsk/c1t2d0s0

echo Checking target disk FS
fsck -y /dev/dsk/c1t2d0s0
fsck -y /dev/dsk/c1t2d0s1
fsck -y /dev/dsk/c1t2d0s3
fsck -y /dev/dsk/c1t2d0s5
fsck -y /dev/dsk/c1t2d0s6
fsck -y /dev/dsk/c1t2d0s7

sync;sync;sync
cd /
umount /mnt
init 5

---------- >8 cut here 8< ----------
Here's my format.txt which gets piped to the format command:
The first line selects disk 1 as the target, if you screw up,
you'll blow away your source disk, soo...  This is for 32gb
disks.  You'll have to modify this to match your setup.

My slice table:  (which likely you'll have to change to your
needs, by editing the file below) :)

s0 /			500M
s1 /usr			2gb
s2 (entire disk)	33gb
s3 /var			2gb
s4 swap			4gb
s5 /opt			10gb
s6 /usr/local		5gb
s7 /export/home		10gb


---------- >8 cut here 8< ----------
1
p

0
root
wm
0
363c

1
usr
wm
363
1452c

3
var
wm
1815
1452c

4
swap
wu
3267
2904c

5
unassigned
wm
6171
7260c

6
usr
wm
13431
3641c

7
home
wm
17072
7548c

p
p
label
y
q
verify
----- >8 cut here 8< -----


Now once you have your cloned disks, you'll have to rename/reip them by
doing the following:

echo newhostname >/etc/hostname.eri0; echo newhostname >/etc/nodename

Then vi /etc/net/*/hosts to edit each of the rpc host files.  

As long as your /etc/hosts file contains all your hosts, you're in good
shape.


Hope this was helpful to someone. :)