2008-06-01: Rare bookstore haul
Last night I went to the Palo Alto Boarder's in person, which is the first
time I've been to a regular bookstore in quite a while. I usually order
all of my books on-line since it's easier to check against lists (and
against what books I have around the apartment). But when I go in person,
I usually discover something being published that I'd not realized was
there (if I managed to stay current on Locus, this probably
wouldn't happen as often).
This time, it was the discovery that Warner is reprinting Octavia Butler's
work in omnibus editions. I snapped up the Patternist omnibus and
almost picked up the Xenogensis omnibus, stopped only because I
already have way more books than I'm going to read any time soon.
Here's the haul:
Iain M. Banks — Matter (sff)
Octavia E. Butler — Seed to Harvest (sff)
Jay Lake — Mainspring (sff)
Ken MacLeod — Newton's Wake (sff)
Geraldine McCaughrean — Peter Pan in Scarlet (sff)
Larry Niven — Ringworld (sff)
I've of course already read Ringworld (several times, in fact), but
the last time was about fifteen years ago and I never owned my own copy.
(Wow, the current cover for the mass market edition is ugly. What is with
the hot pink?)
I placed a Powell's order last night anyway, since the new Kushiel book is
out but isn't in the bookstore yet, and that's not one that I wait long on
buying.
2008-06-01: podlators 2.1.0
Switching to Perl 5.10 for Debian unstable uncovered a bunch of little
problems with the Pod::Man based on Pod::Simple that's included with it.
I've unfortunately been slow to fix those problems due to being buried in
other work, but this release finally cleans up the ones that have been
reported, namely:
- Font problems with two or more C<> blocks in a =head
- Fix subheadings with leading *roff metacharacters
- Fix hyphen escaping to not corrupt eth characters
However, the big change is that pod2man now supports a -u (--utf8) option
saying to not munge input characters into ASCII and *roff escapes that
only work with troff and instead output literal UTF-8 characters and
assume one's man program can cope. This will now be true of Debian as of
lenny.
I have no good ideas for how to do the transition, since the resulting
*roff is not portable and, if copied to another system, can even cause
deficient nroff programs to segfault. One of the goals of Pod::Man was
always to produce portable *roff output, so this option is not the
default. However, it is clearly the Right Thing To Do for systems whose
man and/or nroff are sufficiently intelligent; any other option either
requires embedding groff-specific character escapes (same portability
problem) or corrupting non-ASCII characters badly (the previous approach).
Any bright ideas welcome. In the meantime, people who know they're
working on systems with modern nroff and man can at least get correct
output now.
Hopefully podlators 2.1.0 will make it into lenny (as part of the Perl
package, since it's a Perl core module), either as 5.10.1 if upstream is
fast enough or as a Debian-local modification. (We definitely need a new
Perl release before lenny; #480428
would be embarassing and is already driving me nuts by producing spurious
output from lots of cron jobs. I'm thinking about asking for an upgrade
of the priority.)
2008-06-09: Lintian and Policy updates
I'm a bit slow on the journal-writing front, but belatedly I should
mention here that I did upload the new Policy (3.8.0.0) and then a quick
bug-fix. If you have a trigger-enabled dpkg, you don't see that you're
registering non-existent doc-base files.
Lintian should catch that, but doesn't. Yet.
I've been trying to find time to kick off the next round of Policy
updates. There are several proposals that just need text written for
them: #143941 and
#241333 are obvious at this point,
#416450 should be fairly simple, and
I think the due diligence has been done on
#470994. There are other fairly
straightforward ones too. If anyone beats me to it for any of those (or
even better, other bugs), I won't complain.
I also closed all of the old Policy bugs that we'd tagged wontfix except
for one that I forwarded to the
tech-ctte.
I also uploaded Lintian 1.24.0 on Saturday. This contains the first steps
in a better documented and hopefully faster and cleaner way of extracting
information from the lab. Rather than opening and reading files in every
check that need data, there are now Lintian::Collect modules that can hold
common code and cache the results across multiple different check
scripts. Currently, they're not widely used, but they're now available to
all checks (hence the version bump).
This release also contains some truly fantastic work by Jordà Polo Bardés
on the web site. Look at
lintian.debian.org now. It looks like
a real web site! Jordà is working on a Google Summer of Code project for
Lintian, but it wasn't even the web pages; that was just a bonus. I'm
very happy with the outcome.
Lintian is still above 100 bugs and I want to start expanding the new
infrastructure, so there's more work to be done, but I'm very happy with
the progress lately. And it was fun to spend two days solid working on
Policy and Lintian for a change, instead of lots of different work
projects (or feeling low on energy and barely managing to get a handful of
things done).
2008-06-12: FAI and Ubuntu
My project for the past three days has been to upgrade our etch-based
Debian build system to add support for Ubuntu hardy. We're primarily a
Debian shop, but we want to use Ubuntu for the public timeshare systems
due to its faster stable update cycle. In that environment, the latest
and greatest software and active security support are both important
simultaneously, and server-level stability isn't as vital. Ubuntu wins
over other options because it's basically Debian and therefore shares with
Debian the huge package selection, which is important for university
timeshare systems. (We also use the external R package repository; we
have a lot of heavy R users.)
First, FAI has changed a lot since the etch release and hardy has a far
more recent version of fai-client, which means that just bootstrapping an
Ubuntu NFS root on an etch system and trying to install it with the etch
FAI doesn't work due to the mismatch between the server expectations and
the fai-client and fai-nfsroot packages. FAI during the etch release
boots with a traditional read-only NFS root and works around the fact that
its boot device is read-only. Current FAI does essentially the same trick
that a live CD does, using unionfs to allow local modifications to the NFS
root with a backing tmpfs file system, which is kind of scary but allows
FAI to be much simpler.
The first thing I tried was debootstrapping Ubuntu (using the debootstrap
from backports.debian.org) and then
downgrading its fai-client and fai-nfsroot packages to the etch versions,
but changes in other part of the system (particularly mount) makes that
not very viable.
One of the nice things about FAI is that it's basically a collection of
shell scripts, which means that it's architecture-independent and one can
easily install the unstable packages on an otherwise-etch system. So I
upgraded fai-server to the current Debian sid version of FAI and installed
the sid versions of fai-nfsroot and fai-client into the Ubuntu NFS root
after generating it with make-fai-nfsroot as normal. That works much
better. However, there are a few Ubuntu-specific problems to work around
(which are patched in the Ubuntu version of FAI, but I don't want to have
to track down an Ubuntu system to generate the NFS roots when the rest of
our infrastructure is etch):
Ubuntu hardy ships with a unionfs that can't write to existing files,
only create new files. I'm not sure whose bug this is, possibly
upstream, but etch unionfs doesn't have this problem. The workaround
is to delete the files that FAI wants to overwrite out of the NFS root
after generating it with make-fai-nfsroot, which you can do in a hook
or by wrappering it and doing the deletions afterwards (we do the
latter because we have other local tweaks we make). You want to
delete /etc/resolv.conf, /etc/syslog.conf, and auth.log, syslog,
daemon.log, kern.log, lpr.log, mail.*, user.log, debug, and messages
in /var/log.
Ubuntu hardy defaults to using upstart, which completely fails in an
FAI NFS root environment. I'm not sure why, but first upstart claims
the respawn line of every file in /etc/event.d/* is a syntax
error (?!) and, even if that's fixed, doesn't actually spawn a getty
(at least on a serial console). The implication is that you can't
Ctrl-C at the end of the build and work on the system before
rebooting, which makes diagnosing problems very difficult. I tried a
bunch of different possibilities for an event.d file (and wow is that
ever either underdocumented or obscurely documented —
I've still never found a man page or other document clearly explaining
the possible configuration options) before giving up. The Ubuntu FAI
package forces installation of sysvinit in the NFS root, and that
works fine (with the normal inittab tweaking to spawn a getty on the
serial console). upstart works fine on the newly built system,
including serial console, so I'm not sure why the NFS root is broken.
Other than that, most of the work was upgrading the whole system to using
the new boot method (changing to boot=live on the PXE kernel
options, including live-initramfs in the NFS root, copying the kernel and
initrd out of the NFS root into the tftp area instead of using
fai-kernels, and adding initrd to the PXE kernel options). I'm now
successfully building etch and hardy systems using our FAI setup and will
probably upgrade the production service next week.
There is one other fairly serious Ubuntu design misfeature that is only
tangentially related but that cost me a lot of time. Ubuntu has modified
update-grub to make it use ucf, which breaks the traditional update-grub
semantics in extremely strange and confusing ways that took hours for me
to diagnose.
For those who aren't familiar with the details of how update-grub works,
it uses comments in /boot/grub/menu.lst as a template to generate the
kernel list at the end of menu.lst. It's not the best design, since it
uses the same file as both a template and the active configuration file,
but it works, and has worked the same way for many years. It's simple and
reliable: you only change the defaults in the comments and then
update-grub adds in the kernel-specific information and generates the
active menu for you. If you want to edit the kernel list yourself for
some reason, don't use update-grub; it's completely optional.
To see how using ucf breaks this, remember that ucf's goal is to preserve
local changes to a configuration file, but its notion of local changes is
fundamentally contrary to how menu.lst works. Suppose that you're a large
organization with a lot of systems with a variety of kernel revisions, for
one reason or another, but with a desire to have uniform kernel parameters
like console speed and timeouts. With update-grub, you can just copy your
menu.lst template to all of your systems and run update-grub afterwards,
and update-grub will apply your new defaults and generate a new kernel
list using them. However, with Ubuntu's update-grub, it generates the
initial menu.lst and kernel list, and then when you replace that with a
fresh template, ucf sees that you've removed the kernel list. It thinks
that's a local modification, and when update-grub generates a new kernel
list, it "preserves" your modification and leaves the kernel list out of
the new menu.lst. Tada, instant unbootable system.
Even worse, this bug is sticky: you can re-run update-grub all you want,
it succeeds without errors, and it keeps generating an empty menu.lst.
This is a completely mystifying error until you dig into the details of
what's going on. To recover, you have to run ucf --purge
/var/run/grub/menu.lst (and that path, or for that matter any of the ucf
stuff, is completely undocumented in the update-grub man page) and then
re-run update-grub, at which point ucf will display a debconf prompt
(which doesn't support debconf preseeding, but that's another problem).
If you then select three-way merge (not the default) from that prompt, you
get a correct menu.lst.
I'm not sure why Ubuntu made this change, but as near as I can tell, it's
fundamentally architecturally broken. The specific problem I had was that
we were copying our template over and re-running update-grub in a
post-install script, resulting in a system that always had an empty kernel
menu and wouldn't boot. Once I finally figured out what's going on, I
worked around it by being very careful to put our template in place before
the initial system bootstrap and then never changing menu.lst afterwards,
but this is a bad limitation for a large site that scales system
administrators by automating configuration file updates. But maybe I'm
missing some subtlety. I'll go file a Launchpad bug shortly after posting
this and we'll see what happens.
2008-06-13: Latest haul
Both my Powell's order and my order from Aqueduct Press came in, so I now
have another pile of new books to sort through and choose reading material
from.
Albert-László Barabási
— Linked (non-fiction)
Bruce Bethke — Headcrash (sff)
Jacqueline Carey — Kushiel's Mercy (sff)
L. Timmel Duchamp — Renegade (sff)
L. Timmel Duchamp — Tsunami (sff)
L. Timmel Duchamp — Blood in the Fruit (sff)
L. Timmel Duchamp — Stretto (sff)
Louise Erdrich — The Antelope Wife (sff)
Kelley Eskridge — Solitaire (sff)
Nicola Griffith — With Her Body (sff)
Sarah Hall — Daughters of the North (sff)
Ellen Klages — The Green Glass Sea
(mainstream-ish)
The Jacqueline Carey book was of course the reason for the order. I hope
to read it in the next few days.
I'm not sure if I'll like the Marq'ssan Cycle by Duchamp, but it's the
sort of project that I want to support — feminist literature
published by a small press, with excellent reviews from the likes of
Samuel Delaney. I'm happy to buy the whole series to support the effort
even if I end up not liking it that much.
2008-06-29: General update
It's been a while since I've posted anything except reviews and I'm
overdue for a general update.
With the Kerberos upgrade project finished last month, I'm currently in
the wonderful situation of not having any large active projects at work
and being given time to generally catch up. With luck, I'll have a couple
more months of that before things really heat up. I only have to consult
on a few projects and otherwise can deal with all the operational issues
that I'd been putting off. A lot of my time lately has been spent on
Puppet configuration cleanup and improvements, and there's at least
another solid week of that to go, as well as a pile of overdue
documentation to write.
On the Debian side, Lintian has moved to a new
Git repository and Adam
D. Barratt has started committing directly. Frank has also had time to do
a lot of work, which means that Lintian development is proceeding quite
well without a lot of active attention on my part. That's been wonderful
and has given me time to focus on other things.
Most of my Debian effort of late has gone into the new Shibboleth
packaging team and review of Shibboleth 2.0 packages, the last of which
was uploaded last week and is sitting in NEW. Thanks to Ferenc Wágner for
the fine work! All the packages are now in Git and migrated to Alioth and
there's a wiki page
for the packaging team. Scott Cantor, the primary upstream developer, has
joined the mailing list and has been extremely helpful and responsive.
Sam Hartman has also migrated the krb5 packaging repository to Git and
Alioth, and last Friday I finally finished migrating the openafs
repository. Both are unfortunately rather huge since they contain all the
historic upstream tarballs (extracting that would have been hideously
complicated to do as part of the migration), but they shouldn't grow
nearly as much now that we can use pristine-tar. This is the heart of the
Kerberos and AFS packaging; there's now just kerberos-configs and my two
PAM modules to move (more on the complexities of the PAM module Git
migration in a later post).
On the personal front, I've been travelling a lot lately, but that will
now calm down for a while. I have some company next month, though, and
hopefully will be up for it by then (right now, I just want to hole up in
my apartment and do my own thing and not interact for a bit).
I've been reading George Orwell: An Age Like This (1920-1940), the
first of a four-volume collection of his letters, essays, and reviews. I
was inspired to buy the first volume after reading one of his essays
on-line after a Usenet discussion some time back, and I'm delighted I did.
It took me a bit to get into it, but now I'm finding it absolutely
fascinating. Orwell has a talent for concise and accurate descriptions of
life (primarily of the poor) that capture the emotional tone as well as
the surface details. That's mixed with interesting book reviews (I love
reading book reviews), intriguing tidbits on his writing process and how
he viewed being a writer, and snippets of the day-to-day of his life that
have survived in letters. It's rather like reading a blog, and Orwell is
a good enough writer that even his unparagraphed, abbreviated commentary
in letters is full of memorable turns of phrase or observations.
If you like that sort of thing, I recommend it highly. I'm going to buy
the remaining three volumes and read the whole collection, plus quite
likely pick up Orwell's other books besides 1984 and Animal
Farm.
2008-06-30: Git, Debian, and upstream
I'm in the process of converting all of my software repositories to Git,
but to date I've only converted either Debian packaging repositories for
software I don't personally maintain or repositories for software without
a corresponding Debian package. But next on my list to tackle are
pam-afs-session and pam-krb5, for which I'm both packager and upstream. Currently, the
upstream development is in bzr and the Debian packaging is in svk.
What I used to do, with Subversion, is keep the upstream development and
Debian packaging in the same repository. I'd exclude debian/* when
doing releases and then export it into an unpack of the release tarball to
do Debian package builds. That worked great for me, but had some
significant drawbacks: the Subversion repositories generally weren't
public (I don't want to host upstream development on Alioth), and manual
work is required to get the upstream tarball to use with the Debian
package build.
pristine-tar, which lets you commit a tarball to a Git repository in a
space-efficient manner by basing it off of an upstream branch, is awesome.
It's one of the best ideas in revision control since DVCSes. Debian
package builds can be done using only the information in the repository
without bloating the repository with huge binary blobs. And for upstream
development, I can capture in the repository exactly what I released.
However, for pristine-tar to work properly, I need to have a branch that
represents the full release tarball, including the stuff that I don't want
to normally track in the repository for regular development (generated
files like configure that are created with make dist). I could let all
that stuff be added with an xdelta, but I think that's ugly and won't be
space-efficient. That means that when I combine Debian packaging and
upstream development into one repository, I can't easily use the upstream
development branch as the basis for the Debian packaging and pristine-tar,
since it's missing all those generated files that are part of the tarball
release.
I could use two separate repositories, but that makes it harder to
cherry-pick upstream commits and also requires that I have two separate
directories for working on the same package, which I don't like.
What I'm leaning towards doing at the moment is doing development on
master, importing released tarballs into an upstream branch
that shares no common commits with master, checking in the release
tarballs under both the upstream release name and the Debian .orig.tar.gz
name using pristine-tar based on the upstream branch, and branching
a debian branch off of upstream instead of master.
It's a little weird to have two separate lines of development that share
no common ancestor but have the same files, but it's the solution that
seems to work best for what I want and lets me store space-efficient
tarballs and use git-buildpackage to do Debian builds without needing
anything outside of the repository.
All posts