Embedded x86 Linux

Another Update: Check out the new version without squashfs here.
Updated!  I modified the squashfs and init so now the entire init script is executed properly, before it would work, but we were missing a few things in the final init stage.

DebianThis rough draft howto describes my experience, minus most of the goof-ups, of how you can setup a small x86 embedded system. A typical linux box that boots grub does the following:

  • Read the MBR
  • load the kernel pointed to by the bootloader
  • load the initial ram disk
  • execute an init script who’s main purpose is to set the environment up so that the real root file system can be mounted
  • then it mounts the FS and executes the real init script, but only after the preliminary init mounts the real root file system read/write.

Confused yet? It gets worse if you must use a read only file system like an IDE/CF flash disk. The preliminary init (ran from initial ram disk) must mount the real file system (usually squashfs) read only. It also sets up a read/write FS in RAM so that the system may store and use things later like /proc, /dev, and /var. After creating both read only and read/write file systems (FS) it merges them with the unionfs/aufs kernel function in init. Then it switches root and executes the real init. There fortunately are many “LiveCD” scripts that set this up for you automatically. These init scripts (like casper in Debian) are overkill and can break an embedded system whose environment is known ahead of time. Its a bit like using this toilet paper dispenser in a mission critical application, like after eating the Habenero Hotwings and drinking 5 pints of beer from the local pub.

Also I hate all the autodetecting that comes with most live cds, power glitches and other things can trick it and it is slooowwww, Embedded systems must be reliable! Mine are 500MHz dinosaurs with no electro-mechanical parts whatsoever.

I like to boot the target filesystem root on the target hardware as an actual r/w FS first and set everything up the way I want it. This has the side effect of also testing my hardware configuration and allows me to make major changes quickly.

Then I halt and mount the target on another host to freeze the system into my own read-only install. Remember! It must be readonly if we are to run off of flash in the end, b/c of the limited writes.

Here we go………….

First create skeleton debian install:
Usage: debootstrap [OPTION]… <suite> <target> [<mirror> [<script>]]

% debootstrap sid fs-root-debian-sid

Put this file system on a drive and boot your real hardware. Set everything up the best you can (Minor changes can bee made later, don’t worry).

Back at the host:

chroot to new install or boot the real system hardware using a non flash hard drive:

install squashfs-tools, kernel, and same versions of squashfs and unionfs kernel modules
You may have to fix /etc/resolv.conf for this and other apt based installations
# apt-get install kernel-image-xxx unionfs-modules-xxx squashfs-modules-xxx squashfs-tools

unchroot, do:

Create the boot-dir:

# mkdir -p fs-boot/boot/grub

chroot, do:
# apt-get install grub

make sure mkinitramfs and busybox is installed

unchroot, do:

Copy the boot loader

# cp -a fs-root/usr/lib/grub/i386-pc/* fs-boot/boot/grub/

Copy the kernel over:

$ cp -a fs-root/boot/* fs-boot/boot/

Create the squashed FS

% mksquashfs fs-root fs-squashfs.squashfs -all-root # Makes all files owned by root

On the chrooted target, install initramfs-tools
# apt-get install initramfs-tools

Now the tricky part, edit init scripts, then create the initramfs:

chroot to the fs-root and create our squashfs script:
# vim /usr/share/initramfs-tools/scripts/squashfs

modify it to look like the included file “squashfs_script”, some highlights are:

# This is how we mount the rootfs
# substitute /dev/sda1 for what it will be on the target upon boot, I’m using a usb flashdrive
mount -r -t ext2 /dev/sda1 /.tmpfs/.mnt
losetup /dev/loop0 /.tmpfs/.mnt/boot/fs-squashfs.squashfs
mount -r -t squashfs /dev/loop0 /.tmpfs/.sqfs

# Here we create the final rootfs
mount -w -t unionfs -o dirs=/.tmpfs/.overlay=rw:/.tmpfs/.cdrom=ro:/.tmpfs/.sqfs=ro unionfs /.union

# And switch over to the new FS and run the real init, This is done as the last line of the default debian etch init script, not the squashfs script, you should comment out the old last line:

# Chain to real filesystem
maybe_break init
#exec run-init ${rootmnt} ${init} “$@” <${rootmnt}/dev/console >${rootmnt}/dev/console
exec run-init /.union /sbin/init “$@” </.union/dev/console >/.union/dev/console

Now, still crooted as the target, edit /boot/grub/menu.lst to be:

# menu.lst – See: grub(8), info grub, update-grub(8)
default 0
timeout 5

title Debian live
root (hd0,0)
kernel /boot/vmlinuz vga=791 root=/dev/loop0 ramdisk_size=100000 boot=squashfs #username=user hostname=debian persistent
initrd /boot/file1.img
# Note the boot=squashfs tells init to execute scripts/squashfs
# root=/dev/loop0 is our readonly root squashfs mounted in the initscript
# ramdisk_size=100000 gives the kernel (and us) some space (100M) to work with our unionfs

We’re almost done, now make the initramfs, recall we are still chrooted

# mkinitramfs -o file1.img <kernel-version-here>
please put your kenel version int the appropriate spot above (whatever string is appended to /lib/modules on the target)

Exit from chroot and copy the resulting file1.img to your targets’ boot directory /fs-boot/boot/

# cp file1.img /fs-boot/boot/

Now copy fs-root and fs-boot files to your production device, I’m using a usb flash drive (I assume is partitioned this way):

# cp -a fs-boot/* /mnt/sda1/
# cp -a fs-root/fs-squashfs.squashfs /mnt/sda1/

Make sure that the /proc /dev and /sys directories are empty, they will be filled later by the last stages of init on the unionfs

Install grub to the target device so it will boot, I will not cover that here, its covered everywhere else on the net quite well

I usually like to test the setup under vmware or qemu, as it can save time when debugging.

Comments? Mistakes? Help me make this better. I welcome all non-spam feedback.

4 Responses to “Embedded x86 Linux”

  1. buttle2000 Says:


    Thanks for the tutorial.

    I’m a bit confused with your directory layout.
    fs-root and fs-boot
    Could you explain that a little more?

    In the squahfs script you make reference to a casper directory, but casper isn’t part of your setup, is it?

    Thanks once again.

  2. electronjunkie Says:

    This stuff is confusing, but fun. Thanks for your feedback, I found a few problems: the line:
    % debootstrap sid fs-root-debian-sid
    should be
    % debootstrap sid fs-root

    I also fixed the casper reference in the squashfs script (should be blank). I ripped alot of this off from a casper tutorial.
    fs-root and fs-boot are two separate staging areas for two different purposes. The idea behind fs-root is that it is the final root file system before being made into a squashfs image.
    The fs-boot directory is supposed to contain the files necessary to boot such as the grub stages and configuration. It also contains the initial ramdisk. It must be separate from the root because the device needs it 1st, before the root squashfs is mounted.

    This tutorial may have other bugs, but I still reference it and use it successfully when I create new systems so it is at least close. Do you use qemu or vmware do debug your boot?

  3. jordix Says:

    I tried it using ubuntu Gutsy. Finally I could have it working, but…
    -In the init script, instead of modify the last line, I changed the line
    export rootmnt=/root
    export rootmnt=/.union
    Not doing so caused the script fail at executing init-bottom.
    And I have to stop the script for a moment just before the mount procedure (break=mount). It seems to need some time to finish loading some things before it can execute correctly your squashfs-script.


  4. electronjunkie Says:

    You are right, there are problems with the current script. That is probably one of several ways to fix it.

    I have been meaning to update the squashfs script, but wordpress won’t let me upload or paste certain files like html or text without mangling them. My latest squashfs script has changed quite a bit, but until I can figure out how to post it without wordpress mangling the html, I am stuck with this last version. You can email me at lm317t att gmail dot com if you’d like discussing this further. Replace the att with @ and dot with .

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

<span>%d</span> bloggers like this: