The Symptoms
The problem is that your server will not reboot after a kernel update. The kernel panics with something similar to:
Mounting root filesystem mount: error 19 mounting mount: error 2 mounting none Switching to new root switchroot: mount failed: 22 umount /initrd/dev failed: 2 Kernal panic - not syncing: Attempting to kill init!
I’ve seen then on RHEL and CentOS boxes. This is cause by a bad entry in /etc/fstab. A plesk installer/updater cause this to happen.
LABEL=/ / ext3,usrquota defaults 1 LABEL=/boot /boot ext2 defaults 1 2 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 LABEL=SWAP-hda2 swap swap defaults 0 0
Note the first line has:
LABEL=/ / ext3,usrquota defaults 1
instead of
LABEL=/ / ext3 usrquota,defaults 1 1
The userquota is over against the ext3, essentially defining a fstype that the OS does not understand. Root cannot be mounted and the kernel panics.
The Fix
The fix is relatively simple. Just correct the lines in the /etc/fstab and then remove the latest kernel and re-install. You can fix it manually but using ‘rpm -e’ to remove the failed kernel and then run yum or up2date again to get the new kernel installed.
The Conundrum
I had fixed up many boxes back when this first appeared in 2006. There were some discussions over at the Plesk forum about it. I had nearly forgotten about it until this past round of kernel updates. We had some new clients that had not had their kernels updated since their OS was installed. The odd thing is that they had purchased their servers in 2007. The problem appears that their server provider has not updated their installer procedures to fix the issue.
The Lesson
When you find solutions to odd problems, document them. You may never know when the issue will re-appear. My staff does this well in our internal systems, but I often forget to document the issue.