Posts for June 2008

2008-06-01: Rare bookstore haul

Last night I went to the Palo Alto Boarder's in person, which is the first time I've been to a regular bookstore in quite a while. I usually order all of my books on-line since it's easier to check against lists (and against what books I have around the apartment). But when I go in person, I usually discover something being published that I'd not realized was there (if I managed to stay current on Locus, this probably wouldn't happen as often).

This time, it was the discovery that Warner is reprinting Octavia Butler's work in omnibus editions. I snapped up the Patternist omnibus and almost picked up the Xenogensis omnibus, stopped only because I already have way more books than I'm going to read any time soon.

Here's the haul:

Iain M. Banks — Matter (sff)
Octavia E. Butler — Seed to Harvest (sff)
Jay Lake — Mainspring (sff)
Ken MacLeod — Newton's Wake (sff)
Geraldine McCaughrean — Peter Pan in Scarlet (sff)
Larry Niven — Ringworld (sff)

I've of course already read Ringworld (several times, in fact), but the last time was about fifteen years ago and I never owned my own copy. (Wow, the current cover for the mass market edition is ugly. What is with the hot pink?)

I placed a Powell's order last night anyway, since the new Kushiel book is out but isn't in the bookstore yet, and that's not one that I wait long on buying.

2008-06-01: podlators 2.1.0

Switching to Perl 5.10 for Debian unstable uncovered a bunch of little problems with the Pod::Man based on Pod::Simple that's included with it. I've unfortunately been slow to fix those problems due to being buried in other work, but this release finally cleans up the ones that have been reported, namely:

However, the big change is that pod2man now supports a -u (--utf8) option saying to not munge input characters into ASCII and *roff escapes that only work with troff and instead output literal UTF-8 characters and assume one's man program can cope. This will now be true of Debian as of lenny.

I have no good ideas for how to do the transition, since the resulting *roff is not portable and, if copied to another system, can even cause deficient nroff programs to segfault. One of the goals of Pod::Man was always to produce portable *roff output, so this option is not the default. However, it is clearly the Right Thing To Do for systems whose man and/or nroff are sufficiently intelligent; any other option either requires embedding groff-specific character escapes (same portability problem) or corrupting non-ASCII characters badly (the previous approach).

Any bright ideas welcome. In the meantime, people who know they're working on systems with modern nroff and man can at least get correct output now.

Hopefully podlators 2.1.0 will make it into lenny (as part of the Perl package, since it's a Perl core module), either as 5.10.1 if upstream is fast enough or as a Debian-local modification. (We definitely need a new Perl release before lenny; #480428 would be embarassing and is already driving me nuts by producing spurious output from lots of cron jobs. I'm thinking about asking for an upgrade of the priority.)

2008-06-09: Lintian and Policy updates

I'm a bit slow on the journal-writing front, but belatedly I should mention here that I did upload the new Policy ( and then a quick bug-fix. If you have a trigger-enabled dpkg, you don't see that you're registering non-existent doc-base files.

Lintian should catch that, but doesn't. Yet.

I've been trying to find time to kick off the next round of Policy updates. There are several proposals that just need text written for them: #143941 and #241333 are obvious at this point, #416450 should be fairly simple, and I think the due diligence has been done on #470994. There are other fairly straightforward ones too. If anyone beats me to it for any of those (or even better, other bugs), I won't complain.

I also closed all of the old Policy bugs that we'd tagged wontfix except for one that I forwarded to the tech-ctte.

I also uploaded Lintian 1.24.0 on Saturday. This contains the first steps in a better documented and hopefully faster and cleaner way of extracting information from the lab. Rather than opening and reading files in every check that need data, there are now Lintian::Collect modules that can hold common code and cache the results across multiple different check scripts. Currently, they're not widely used, but they're now available to all checks (hence the version bump).

This release also contains some truly fantastic work by Jordà Polo Bardés on the web site. Look at now. It looks like a real web site! Jordà is working on a Google Summer of Code project for Lintian, but it wasn't even the web pages; that was just a bonus. I'm very happy with the outcome.

Lintian is still above 100 bugs and I want to start expanding the new infrastructure, so there's more work to be done, but I'm very happy with the progress lately. And it was fun to spend two days solid working on Policy and Lintian for a change, instead of lots of different work projects (or feeling low on energy and barely managing to get a handful of things done).

2008-06-12: FAI and Ubuntu

My project for the past three days has been to upgrade our etch-based Debian build system to add support for Ubuntu hardy. We're primarily a Debian shop, but we want to use Ubuntu for the public timeshare systems due to its faster stable update cycle. In that environment, the latest and greatest software and active security support are both important simultaneously, and server-level stability isn't as vital. Ubuntu wins over other options because it's basically Debian and therefore shares with Debian the huge package selection, which is important for university timeshare systems. (We also use the external R package repository; we have a lot of heavy R users.)

First, FAI has changed a lot since the etch release and hardy has a far more recent version of fai-client, which means that just bootstrapping an Ubuntu NFS root on an etch system and trying to install it with the etch FAI doesn't work due to the mismatch between the server expectations and the fai-client and fai-nfsroot packages. FAI during the etch release boots with a traditional read-only NFS root and works around the fact that its boot device is read-only. Current FAI does essentially the same trick that a live CD does, using unionfs to allow local modifications to the NFS root with a backing tmpfs file system, which is kind of scary but allows FAI to be much simpler.

The first thing I tried was debootstrapping Ubuntu (using the debootstrap from and then downgrading its fai-client and fai-nfsroot packages to the etch versions, but changes in other part of the system (particularly mount) makes that not very viable.

One of the nice things about FAI is that it's basically a collection of shell scripts, which means that it's architecture-independent and one can easily install the unstable packages on an otherwise-etch system. So I upgraded fai-server to the current Debian sid version of FAI and installed the sid versions of fai-nfsroot and fai-client into the Ubuntu NFS root after generating it with make-fai-nfsroot as normal. That works much better. However, there are a few Ubuntu-specific problems to work around (which are patched in the Ubuntu version of FAI, but I don't want to have to track down an Ubuntu system to generate the NFS roots when the rest of our infrastructure is etch):

Other than that, most of the work was upgrading the whole system to using the new boot method (changing to boot=live on the PXE kernel options, including live-initramfs in the NFS root, copying the kernel and initrd out of the NFS root into the tftp area instead of using fai-kernels, and adding initrd to the PXE kernel options). I'm now successfully building etch and hardy systems using our FAI setup and will probably upgrade the production service next week.

There is one other fairly serious Ubuntu design misfeature that is only tangentially related but that cost me a lot of time. Ubuntu has modified update-grub to make it use ucf, which breaks the traditional update-grub semantics in extremely strange and confusing ways that took hours for me to diagnose.

For those who aren't familiar with the details of how update-grub works, it uses comments in /boot/grub/menu.lst as a template to generate the kernel list at the end of menu.lst. It's not the best design, since it uses the same file as both a template and the active configuration file, but it works, and has worked the same way for many years. It's simple and reliable: you only change the defaults in the comments and then update-grub adds in the kernel-specific information and generates the active menu for you. If you want to edit the kernel list yourself for some reason, don't use update-grub; it's completely optional.

To see how using ucf breaks this, remember that ucf's goal is to preserve local changes to a configuration file, but its notion of local changes is fundamentally contrary to how menu.lst works. Suppose that you're a large organization with a lot of systems with a variety of kernel revisions, for one reason or another, but with a desire to have uniform kernel parameters like console speed and timeouts. With update-grub, you can just copy your menu.lst template to all of your systems and run update-grub afterwards, and update-grub will apply your new defaults and generate a new kernel list using them. However, with Ubuntu's update-grub, it generates the initial menu.lst and kernel list, and then when you replace that with a fresh template, ucf sees that you've removed the kernel list. It thinks that's a local modification, and when update-grub generates a new kernel list, it "preserves" your modification and leaves the kernel list out of the new menu.lst. Tada, instant unbootable system.

Even worse, this bug is sticky: you can re-run update-grub all you want, it succeeds without errors, and it keeps generating an empty menu.lst. This is a completely mystifying error until you dig into the details of what's going on. To recover, you have to run ucf --purge /var/run/grub/menu.lst (and that path, or for that matter any of the ucf stuff, is completely undocumented in the update-grub man page) and then re-run update-grub, at which point ucf will display a debconf prompt (which doesn't support debconf preseeding, but that's another problem). If you then select three-way merge (not the default) from that prompt, you get a correct menu.lst.

I'm not sure why Ubuntu made this change, but as near as I can tell, it's fundamentally architecturally broken. The specific problem I had was that we were copying our template over and re-running update-grub in a post-install script, resulting in a system that always had an empty kernel menu and wouldn't boot. Once I finally figured out what's going on, I worked around it by being very careful to put our template in place before the initial system bootstrap and then never changing menu.lst afterwards, but this is a bad limitation for a large site that scales system administrators by automating configuration file updates. But maybe I'm missing some subtlety. I'll go file a Launchpad bug shortly after posting this and we'll see what happens.

2008-06-13: Latest haul

Both my Powell's order and my order from Aqueduct Press came in, so I now have another pile of new books to sort through and choose reading material from.

Albert-László Barabási — Linked (non-fiction)
Bruce Bethke — Headcrash (sff)
Jacqueline Carey — Kushiel's Mercy (sff)
L. Timmel Duchamp — Renegade (sff)
L. Timmel Duchamp — Tsunami (sff)
L. Timmel Duchamp — Blood in the Fruit (sff)
L. Timmel Duchamp — Stretto (sff)
Louise Erdrich — The Antelope Wife (sff)
Kelley Eskridge — Solitaire (sff)
Nicola Griffith — With Her Body (sff)
Sarah Hall — Daughters of the North (sff)
Ellen Klages — The Green Glass Sea (mainstream-ish)

The Jacqueline Carey book was of course the reason for the order. I hope to read it in the next few days.

I'm not sure if I'll like the Marq'ssan Cycle by Duchamp, but it's the sort of project that I want to support — feminist literature published by a small press, with excellent reviews from the likes of Samuel Delaney. I'm happy to buy the whole series to support the effort even if I end up not liking it that much.

2008-06-29: General update

It's been a while since I've posted anything except reviews and I'm overdue for a general update.

With the Kerberos upgrade project finished last month, I'm currently in the wonderful situation of not having any large active projects at work and being given time to generally catch up. With luck, I'll have a couple more months of that before things really heat up. I only have to consult on a few projects and otherwise can deal with all the operational issues that I'd been putting off. A lot of my time lately has been spent on Puppet configuration cleanup and improvements, and there's at least another solid week of that to go, as well as a pile of overdue documentation to write.

On the Debian side, Lintian has moved to a new Git repository and Adam D. Barratt has started committing directly. Frank has also had time to do a lot of work, which means that Lintian development is proceeding quite well without a lot of active attention on my part. That's been wonderful and has given me time to focus on other things.

Most of my Debian effort of late has gone into the new Shibboleth packaging team and review of Shibboleth 2.0 packages, the last of which was uploaded last week and is sitting in NEW. Thanks to Ferenc Wágner for the fine work! All the packages are now in Git and migrated to Alioth and there's a wiki page for the packaging team. Scott Cantor, the primary upstream developer, has joined the mailing list and has been extremely helpful and responsive.

Sam Hartman has also migrated the krb5 packaging repository to Git and Alioth, and last Friday I finally finished migrating the openafs repository. Both are unfortunately rather huge since they contain all the historic upstream tarballs (extracting that would have been hideously complicated to do as part of the migration), but they shouldn't grow nearly as much now that we can use pristine-tar. This is the heart of the Kerberos and AFS packaging; there's now just kerberos-configs and my two PAM modules to move (more on the complexities of the PAM module Git migration in a later post).

On the personal front, I've been travelling a lot lately, but that will now calm down for a while. I have some company next month, though, and hopefully will be up for it by then (right now, I just want to hole up in my apartment and do my own thing and not interact for a bit).

I've been reading George Orwell: An Age Like This (1920-1940), the first of a four-volume collection of his letters, essays, and reviews. I was inspired to buy the first volume after reading one of his essays on-line after a Usenet discussion some time back, and I'm delighted I did. It took me a bit to get into it, but now I'm finding it absolutely fascinating. Orwell has a talent for concise and accurate descriptions of life (primarily of the poor) that capture the emotional tone as well as the surface details. That's mixed with interesting book reviews (I love reading book reviews), intriguing tidbits on his writing process and how he viewed being a writer, and snippets of the day-to-day of his life that have survived in letters. It's rather like reading a blog, and Orwell is a good enough writer that even his unparagraphed, abbreviated commentary in letters is full of memorable turns of phrase or observations.

If you like that sort of thing, I recommend it highly. I'm going to buy the remaining three volumes and read the whole collection, plus quite likely pick up Orwell's other books besides 1984 and Animal Farm.

2008-06-30: Git, Debian, and upstream

I'm in the process of converting all of my software repositories to Git, but to date I've only converted either Debian packaging repositories for software I don't personally maintain or repositories for software without a corresponding Debian package. But next on my list to tackle are pam-afs-session and pam-krb5, for which I'm both packager and upstream. Currently, the upstream development is in bzr and the Debian packaging is in svk.

What I used to do, with Subversion, is keep the upstream development and Debian packaging in the same repository. I'd exclude debian/* when doing releases and then export it into an unpack of the release tarball to do Debian package builds. That worked great for me, but had some significant drawbacks: the Subversion repositories generally weren't public (I don't want to host upstream development on Alioth), and manual work is required to get the upstream tarball to use with the Debian package build.

pristine-tar, which lets you commit a tarball to a Git repository in a space-efficient manner by basing it off of an upstream branch, is awesome. It's one of the best ideas in revision control since DVCSes. Debian package builds can be done using only the information in the repository without bloating the repository with huge binary blobs. And for upstream development, I can capture in the repository exactly what I released.

However, for pristine-tar to work properly, I need to have a branch that represents the full release tarball, including the stuff that I don't want to normally track in the repository for regular development (generated files like configure that are created with make dist). I could let all that stuff be added with an xdelta, but I think that's ugly and won't be space-efficient. That means that when I combine Debian packaging and upstream development into one repository, I can't easily use the upstream development branch as the basis for the Debian packaging and pristine-tar, since it's missing all those generated files that are part of the tarball release.

I could use two separate repositories, but that makes it harder to cherry-pick upstream commits and also requires that I have two separate directories for working on the same package, which I don't like.

What I'm leaning towards doing at the moment is doing development on master, importing released tarballs into an upstream branch that shares no common commits with master, checking in the release tarballs under both the upstream release name and the Debian .orig.tar.gz name using pristine-tar based on the upstream branch, and branching a debian branch off of upstream instead of master. It's a little weird to have two separate lines of development that share no common ancestor but have the same files, but it's the solution that seems to work best for what I want and lets me store space-efficient tarballs and use git-buildpackage to do Debian builds without needing anything outside of the repository.

Last modified and spun 2017-02-20