Debian fails to boot: libm.so.6 is missing in initrd

After migrating my two identical server systems (radix, vicari) from 686 to amd64 (both Debian squeeze) I encountered the following problem at the next kernel update: radix worked fine whereas vicari failed to boot:

boot/bin/sh: error while loading shared libraries: libm.so.6: cannot open shared object file: No such file or directory

The problem was obviously the initrd. Fortunately I have a serial console on this server (including BIOS access) but Murphy was with me: It took me a few hours to boot a rescue grml successfully over NFS with PXE.

I compared the initrd of both systems:

scp root@radix:/boot/initrd.img-2.6.32-5-openvz-amd64 radix
scp vicari@radix:/boot/initrd.img-2.6.32-5-openvz-amd64 vicari
cd radix; gzip -dc initrd.img-2.6.32-5-openvz-amd64 | cpio -id; cd..
cd vicari; gzip -dc initrd.img-2.6.32-5-openvz-amd64 | cpio -id; cd..
diff -ur radix vicari >diffs

The difference was indeed that libm.so.6 was found in lib for the (working) radix case and in lib64 in the (broken) vicari case. The difference on both systems was /etc/ld.so.config: radix has /lib as first entry whereas vicari had /lib64 as first entry. I modified /etc/ld.so.config to read something like this:

/lib
include /etc/ld.so.conf.d/*.conf
/lib64
/usr/lib64
/usr/X11R6/lib64

and rebuilt the initrd with

dpkg-reconfigure linux-image-2.6.32-5-openvz-amd64

The next boot was fine ...

---

grub2 has a console: Set it to serial if it is not done yet:

serial --speed=57600 --unit=0 --word=8 --parity=no --stop=1
terminal_input console serial
terminal_output console serial

I could boot the system first with the correct initrd provided by radix:

set root='(/dev/sda,msdos1)'
search --no-floppy --fs-uuid --set ef1c09ac-8e3c-4c5d-80c9-313c334ab2ed
linux   /boot/vmlinuz-2.6.32-5-openvz-amd64 root=UUID=ef1c09ac-8e3c-4c5d-80c9-313c334ab2ed ro console=ttyS0,57600n8
initrd  /boot/initrd.img-2.6.32-5-openvz-amd64.WORKS

For the booting from PXE I first had the problem that booting the kernel and TFTP worked but afterwards no other communication. It took me roughly two hours to find out why: The interface on the switch was configured to use tagged 802.1q frames! To switch them to an ordinary port, I used (also from serial console) with a SG-3424:

configure
interface ethernet 24
switchport type access
switchport pvid 2
end