Run the entire cluster environment in a chroot

[geofft]

We made lots of progress in #97 by switching to faster chroots, but for various reasons we probably should be running the entire environment, from mounting the root filesystem in the initrd until shutdown, in a aufs snapshot chroot. The advantages include potential speed improvements by not having to set up the chroot at login and reliability improvements by forcing a reboot to flush out any session changes; in addition, we get to use things like PackageKit? safely because there effectively isn't a concept of things "outside the chroot" any more, and we have fewer hacks for things like CUPS.

In order to make this work, we need to

• figure out how to make reboots sufficiently fast. There are some daemons that can probably just be killed or abandoned without a clean shutdown process, given that they're not writing their state anywhere permanent, but things like AFS still should be shutdown cleanly.
• figure out how to do updates. One option is to set an update flag somewhere outside the chroot, and on next boot, boot into a special update session. Another one is to do the update in _another_ chroot, and use the brsync tool from aufs-tools to flush changes from that chroot to the on-disk branch.

This is a fairly long-term goal. I'm milestoning it as summer 2010 for now so we have an idea of when to look into this heavily, but there's no specific reason for that particular timeframe.

[broder]

Replying to geofft:

• figure out how to make reboots sufficiently fast.

We can also always hope that the work done making bootups fast in Karmic will help us here - their target of 10 seconds to reboot would be more than acceptable for us.

[geofft]

So the interesting problem here is how to take updates without doing so in a chroot.

I can think of two general classes of solution here:

• Run schroot inside the initrd and use that as the "real" root (which should work very smoothly, and is generally desirable). If you're at the login screen and decide you want to take updates, reboot yourself with a kernel command-line parameter. If the initrd detects this parameter, don't run schroot or start up the desktop, just boot to athena-auto-update && reboot.
• Don't start up the desktop automatically, instead boot to a screensaver via Plymouth or somesuch. If a user wakes up the computer, then schroot and start up the desktop from inside it, which should take about two seconds these days. Otherwise, the screensaver periodically checks for updates.