[geeks] Policy for system / package upgrades in Enterprise (LONG)

[gsm at mendelson.com]

On Thu, Jul 29, 2010 at 03:10:44PM +0530, Katrina Gawas wrote:
>Thanks for the reply. We are open to suggestions regarding change of
>OS. U have already mentioned CENTOS & RHEL; What would you recommend
>keeping our requirement of adhering to ISO 27k requirements?

I think I mentioned I had two UBUNTU 9.10 systems, last night I updated 
one of them. I am sorry to admit it is an 800mHz Pentium III with 324m of RAM,
and a 40g hard drive, but it sloggs away running asterisk and downloading 
files from various http servers using jDownloader. It's actually in my
basement and I get into it using ssh and VNC.

Being hardware poor, I was upgrading it to an 1800mHz AMD system with
a whopping 512m of RAM and a 250g hd. The hard drive had spent the last
3 years running 24/6 on my wife's computer. 

She just got an upgrade to a newer computer and so I got the hard drive. Not
the computer, that goes to my son. 

So the first thing I do is to backup using tar the system. This is the one
that GNOME stopped working after an update.

So I booted my UBUNTU 10.04 disk, partitioned and formatted the drive, 
copied over the tar file and then untared it. I made the mistake of
putting it in what was to be /tmp on the new drive. UBUNTU clears /tmp
every boot.

I fixed the necessary files to prevent my DHCP server and asterisk from
starting and changed the hostname and eth0 startup to DHCP. I fixed /etc/fstab
because the drives were in different places and grub's menu.lst.

I installed grub.

It seems that UBUNTU has become too much like Windows 7, upgrades can only
be run from running systems. :-( and to make things worse, only on the
alternate CD, if you download the regular CD, you can only do a wipe and 
install. 

You can do a diskless upgrade (recommended, with no unrecommeded choices
mentioned on their upgrade page), but that requires 1.5g of downloads.
Actually not a problem here after 1am, but during the day and into the night,
it would take until 3am. 

I really did not want to boot and fix the system, but I had no choice. 

I booted it.

I get file not found. :-(

After a web search I found out that there is GRUB 1 which uses menu.lst
and GRUB 2 which uses an incompatible grub.cfg. 9.10 uses GRUB 1, 10.04 
includes GRUB 2. 

I STFW for a GRUB 2 config file after booting from the CDROM and install it. 

It booted the 9.10 system.

I can't upgrade because although there is only 1 ethernet interface on
the computer, it's not the old one, so it now thinks its eth1. I had to
modify /etc/network/intefaces to tell it to use DHCP for eth1.

Then I did the upgrade.

The upgrade downloaded 1.5g of files and proceeded to install them. At that
point I went to sleep. 

When I got up in the morning, the hard drive had died. It no longer would
spin up. I later checked with Western Digital, it had a 3 year, one month 
warranty and it was 3 years and 6 months old . :-(

So I put in a 40g drive and went through the whole process again. While the
copy of the tar file was going on, I downloaded the alternate CD image.

It got it booting again, and now with no changes, except hostname, GNOME
starts to work.

I copied the cd image to my home directory (no tmp this time), mounted
the image and ran the upgrade. 

After about 6 hours of copying, downloading and installing, it finishes.

It goes to reboot and boots the kernel and stops with a command prompt
with BusyBox as its shell. 

WTF.

To make a long story short, it installed many kernels, but the last one was
broken. I think that the person doing their kernel compiles tests it on
his (or her) machine and if it boots, it gets released. 

Plus the installer had recognized I had a grub version 1 menu.lst file and
upgraded it and re-installed grub version 1. The fact that I had a grub
version 2 config file was ignored. Too bad, it pulled a UUID (file system
serial number) out of the air and used it for my root file systems.

Since that UUID did not exist, it would never boot.

I fixed that (each fix requires a boot from the live CD and then a boot
from the hard drive to see if it works).

I also had to choose an older kernel because the default one was both
broken and the initial RAMDISK (with all the drivers) was incomplete.

Not much happens because now you can't just tell it to use eth1, you
have to tell it that it is the primary interface. I don't know what
happens if you have two ethernet ports. :-(

With that fixed, I rebuilt the RAMDISK for the default kernel and rebooted.

Success. Unfortunately only once. The second time I booted it, the RAMDISK
was not loading properly. 

I changed grub.cfg to boot the second kernel in the list by default, but
I live in fear that an update will re-install the broken one.

Geoff.
-- 
Geoffrey S. Mendelson N3OWJ/4X1GM
To help restaurants, as part of the "stimulus package", everyone must order 
dessert. As part of the socialized health plan, you are forbidden to eat it. :-)