How to run Ubuntu and Xenomai on the Freescale IceCube (Lite5200)

I got a Freescale IceCube board (a.k.a. Lite5200) and the task to make Linux run on it with working CAN drivers, Ethernet and USB. Well, since the MPC5200 is a neat little chip with a powerful embedded PowerPC 603e core, I thought, why not run the same as I have on my desktop?
Let's see... 64Mbyte RAM... I think they fell a little short on that part, that'll be no fancy Gnome or KDE desktops :-( But... xfce might just run fine, so we'll go with Xubuntu.
The kernel would have to support big part of the on-chip peripherals, like the FEC (that includes bestcomm DMA), USB-hc and MSCAN. Here's a little guide of what I did, and what the pitfalls were:
First a list of tools needed for this task:

1. A cross-development toolchain. I prefer DENX ELDK.
2. Git and Cogito
3. debootstrap
4. a suitable tftp-server pointing to /tftpboot
   
5. a suitable NFS server exporting the rootfs directory we are going to make
   

1.- Preparing the environment:

On the development host machine do this (better place this in a handy script):

$ export PATH=location_of_eldk/usr/bin:$PATH
$ export CROSS_COMPILE=ppc_6xx-
$ export ARCH=ppc
$ sudo mkdir /tftpboot/

2.- Installing the latest u-boot bootloader:

The IceCube came into my hands with an old version of u-boot (pre 1.1.2), which contained a bug in the ethernet driver causing a tftp download to take very long time, so first thing was to install a newer version in order to not get crazy while waiting for the board to boot off the network:

$ git clone git://www.denx.de/git/u-boot.git u-boot
$ cd u-boot
$ cg-seek U-Boot_1_1_5
$ make Lite5200_LOWBOOT_config
$ make
$ cp u-boot.bin /tftpboot/u-boot-lite5200.bin

Then, on the u-boot prompt:

=> tftp 1000000 u-boot-lite5200.bin
=> protect off ff000000 ff03ffff
=> erase ff000000 ff03ffff
=> cp.b 1000000 ff000000 $(filesize)
=> reset

Now we have a working u-boot. I suppose right now everybody already knows how to configure u-boot correctly (hint: you'll need a valid MAC address). Just in case, here's a little summary:

=> set ethaddr aa:bb:cc:dd:ee:ff
=> set ipaddr 1.2.3.4
=> set serverip 2.3.4.5
=> set netmask 255.255.255.0
=> set bootfile vmlinux-2.6.16-lite5200.UBoot
=> set bootargs console=ttyPSC0,115200 rw panic=1 root=/dev/nfs 
ip=1.2.3.4:2.3.4.5 nfsroot=2.3.4.5/home/yope/lite5200/rootfs
=> set bootdelay 5
=> set bootcmd tftp 1000000\; bootm 1000000
=> save

Of course we still don't have a kernel image to boot from, let alone a working rootfs (actually we should have one, since we installed ELDK, didn't we?). Sylvain Munaut seems to have the right kernel to choose for linux on MPC5200 on his website. Is he the official maintainer of MPC5200 related stuff in the 2.6 kernel? Who knows? It looks like that at least.

3.- Compiling a linux kernel

First I started straight forward following the instructions on Sylvain's page to obtain the sources of kernel version 2.6.16-rc1. It came to my attention that this was slightly outdated, and the git-tree was last modified 8 months ago. Nevertheless I moved on, the typical way:

$ make menuconfig
$ make uImage
$ cp arch/ppc/boot/images/uImage /tftpboot/vmlinux-2.6.16-lite5200.UBoot

Now we have a kernel.

4.- Creating the ubuntu rootfs

This one is a little more complex. I don't have a PowerPC based host, so debootstrap will only do half the work:

$ /usr/sbin/debootstrap --arch powerpc dapper rootfs
http://nl.archive.ubuntu.com/ubuntu/

After a while debootstrap will stop, not being able to run any executable in the new rootfs. That sounds fair to me, since they are all powerpc binaries and my PC is an AMD64 machine. At least debootstrap has downloaded the basic stuff and put a lot of essential deb packages in rootfs/var/cache/apt/archives/. There's also dpkg already installed. So let's boot and see. Pushing the reset button on the IceCube starts booting from the network. Everything seems to work, the kernel displays a lot of boot messages, init is started, but... "can't open initial console.". Bummer. Most probably some devicefiles are missing in rootfs/dev/.

$ sudo mknod rootfs/dev/console c 5 1
$ sudo chmod 600 rootfs/dev/console

This should do the trick. Let's try again. Better this time, but init can't find it's /etc/inittab. Guess we have to create one. I just copied my own inittab as a start, and replaced the line

~~:S:wait:/sbin/sulogin

with

~~:S:wait:/bin/bash

and set the default runlevel to "s", since we most probably don't have a working password database, and just need a shell to start doing things. After rebooting once again, the system came to a prompt. Now, first the usual stuff to get going:

# mount sysfs /sys -t sysfs
# mount proc /proc -t proc
# route add default gw 3.4.5.6

And then try installing all the packages, debootstrap has downloaded for us. This is a lot of trial and error, to solve dependencies and get up to the point where we have a working "apt" to make life easier for us. To start with:

# cd /var/cache/apt/archives/
# dpkg -i *.deb

This will get quite a lot of stuff unpacked, and some of it even configured, but most of it will just give a lot of errors. It's a bit like shooting in the crowd. Now we basically iterate through "dpkg --ignore-depends=XXX -i YYY" and "dpkg --configure --pending" to install everything that leads the shortest way to "apt". The basic rule is that dependencies on XXX can be ignored if XXX is unpacked but cannot be configured yet. After a while we will eventually be able to do:

# apt-get update
# apt-get -f install

to fix all the mess we left brute-forcing dpkg. Now we can try some more fun stuff:

# apt-get install xubuntu-desktop joe gedit mc ...
# joe /etc/X11/gdm/gdm-cdd.conf

Or use your favorite editor, in the section "[security]" set "AllowRemoteAutoLogin=true" and in "[xdmcp]" set "Enable=true".

# joe /etc/inittab

Undo the changes we made earlier. Press reset and reboot.
Now we should be able to run a remote X session on our IceCube!

5.- Getting Xenomai running

Ok, enough playing. Let's have a look at Xenomai. Browsing the SVN repository we can see that there's a new adeos-ipipe patch for powerpc kernel version 2.6.18! Ooops. The previous one is for 2.6.14. Double ooops (we have 2.6.16-rc1, remember?).
This is what I propose (on our development host):

$ git pull git://www.denx.de/git/linux-2.6-denx.git
$ cg-seek e478bec0ba0a83a48a0f6982934b6de079e7e6b3

I wonder whether this makes much sense. Anyway, this is the commit ID of version 2.6.18. I did it this way, hoping to not have to clone another complete git repository, and just get what's missing from where we are until the latest official kernel (or what DENX had from it). I chose DENX git instead of kernel.org, because I hoped that maybe DENX had merged some of the non-official MPC5200 stuff already. In fact, they did, but not until somewhere near 2.6.19. Arghh! That means we'll have to backport some stuff from 2.6.19-rcsomething back to 2.6.18, or try to port Xenomai to the latest 2.6.19.
What do we have now? Let's see.... kernel 2.6.18 without support for MPC5200's internal FEC, among other missing features. FEC support is just around the corner, one version ahead of us, shouldn't be that hard to get it in here...
Now we can browse the DENX git tree (gitweb) to find out the exact commit ID's that introduced "bestcomm DMA" and "MPC5200 FEC" drivers, write them down, and make patches with "cg-mkpatch -r...". Then apply these patches, and hope for the best.
(Note: at the time of writing this, the DENX website is down for repair, so I can't check the git id's). Patching didn't work flawlessly, because of some of the ongoing changes in the kernel to move stuff from arch/ppc to arch/powerpc. That means, I had to correct some paths so "cg-patch" would find the correct files to patch.
After a test run, it seems that this "frankenstein" kernel actually works. It successfully boots from an NFS mounted rootfs, so FEC aught to work. Nice.
Now we have to install Xenomai. Follow the basic instructions:

$ svn co svn://svn.gna.org/svn/xenomai/trunk xenomai
$ cd xenomai
$ scripts/prepare-kernel --linux=where_our_kernel_is 
--adeos=ksrc/arch/powerpc/patches/adeos-ipipe-2.6.18-ppc-1.4-01.patch
--arch=ppc

Now we can build our new kernel with xenomai:

$ make menuconfig

Select "Real-time sub-system -->" and adjust it to your likening. Don't forget to select "Drivers/CAN drivers/RT-Socket-CAN/MSCAN driver for MPC5200" ;-)

$ make uImage
$ cp arch/ppc/boot/images/uImage /tftpboot/vmlinux-2.6.18-lite5200.UBoot

Don't forget to change the "bootfile" variable in u-boot to point to our new kernel.
After that, continue installing the rest of xenomai:

$ ./configure --build=i686 --host=powerpc-linux 
--program-transform-name=s-/usr/-/where_our_rootfs_is/rootfs/usr/- 
--with-cc=ppc_6xx-gcc
$ make

Well, either the option "--program-transform-name" didn't work or I didn't interpret its meaning correclty, anyway, after "make install" files still get installed in "/usr/xenomai"... duh. Just do this:

$ sudo make install
$ sudo mv /usr/xenomai /where_rootfs_is/rootfs/usr/

After this we can reboot once again, and everything should just work... CAN, USB, Ethernet and... Xubuntu! Only thing that's still missing IMHO, is the IDE driver. That's homework ;-)