Posts for January 2012

2012-01-01: 2011 Book Reading in Review

For the year of 2011, I finished and reviewed 60 books. This is a huge milestone for me; it's the first time since the second year I started doing this that the number of books I read actually increased. This gives me more confidence that I've stabilized the year-by-year decline in my reading. I did that while substantially increasing the amount of time I spent enjoying video games, which was another major goal of the year.

Only two books received a 10 out of 10 this year, one fiction and one non-fiction. The novel was Jo Walton's Among Others: the best book I read this year. It's a delightful look at the process of finding a place for oneself in the world and features one of the best protagonists that I've seen.

The non-fiction book was Rory Stewart's The Prince of the Marshes, which means that both of Stewart's books that I've read have received 10 ratings. The Prince of the Marshes is his look at his time spent in the provisional government of Iraq after the overthrow of Saddam Hussein. I think it should be required reading for anyone expressing an opinion on what the US and other western powers should or should not have done in Iraq. It lays bare the difficulties, confusion, and frequent stupidity of going into someone else's country and trying to fix it, and I think seriously calls into question whether this sort of international intervention can ever work.

Other fiction highlights of the year were Kelley Eskridge's Solitaire, a startling and deep look at identity and social connection, and Mira Grant's Feed, a zombie apocalypse story that completely overcame my deep dislike of zombie apocalypse stories. Feed should have won the Hugo in 2011, despite some unbelievable politics and a bit too much cheering for bloggers. This was the year for excellent protagonists, with all three of my top-rated fiction books featuring unique and memorable characters who left a deep and lasting emotional impact.

The two other non-fiction standouts were both a bit dry, but if you have the patience and attention, they reward persistance. Richard Hofstadter's Anti-Intellectualism in American Life is a deserved classic that gave me an eye-opening perspective on the long history of interactions between intellectualism and populism in US politics and culture. David Levering Lewis's God's Crucible is a wonderful history of Islam as it related to Europe and filled in some large gaps in my knowledge of world and religious history.

60 books a year, or five books a month, feels like a comfortable and sustainable level, although I'm going to keep my formal goal at a book a week (52 in the year) to give myself some leeway to either get distracted by video games or by other projects. My reading did concentrate more than usual in science fiction and fantasy this year, and I'd like to add more mainstream fiction and more non-fiction.

The full analysis includes some additional personal reading statistics, probably only of interest to me.

2012-01-04: Miscellaneous mini-haul

Collecting a few random books I've bought over the last couple of months that didn't warrant a full post in their own right.

Max Brooks — World War Z (sff)
Lawrence Lessig — The Future of Ideas (non-fiction)
James Lipton — An Exaltation of Larks (non-fiction)
Vandana Singh — The Woman Who Thought She Was a Planet and Other Stories (sff)
Jack Vance — Tales of the Dying Earth (sff)

I subject everyone else to these because, in the absence of a book database, this is how I keep track of what I already own. It's already been valuable several times. (I really do want to write a book database at some point, though. I keep being tempted to use it as an experiment in learning Java.)

2012-01-07: kstart 4.1

There were a couple of major bugs with k5start -H in 4.0, one reported by pod and one that I ran into myself, so I wanted to get a new release out soon. One fix is for k5start -H and krenew -H when the ticket cache is not renewable; both would incorrectly attempt to reauthenticate even if the remaining lifetime was fine. The other is for k5start -H for a principal different than the ticket cache. k5start wasn't checking the principal, only the ticket lifetime.

But I didn't want to release a new version without implementing a few more things on the TODO list, so this release also has some other features and bug fixes. k5start and krenew when run as daemons now remove their PID files when killed with SIGHUP or SIGTERM. There's a new -s option to krenew telling it to kill a command it's running with SIGHUP when krenew exits (such as when the cache runs out of renewable ticket lifetime). And when k5start or krenew, running as daemons, encounter errors in refreshing the cache, they now retry every minute until the error is resolved.

You can get the latest version from the kstart distribution page.

2012-01-07: Debian and popularity

In the course of a discussion on debian-project, statistics about how many computers run Debian came up. This is a metric that always bothers me a little, and this time I tried to say something about why. It feels like a comment that should have an independent existence outside of that thread, so I'll post it here as well.

One of the delightful things about Debian is that the project consists of a group of people who are working together to create something that, primarily, we all want to use. Making it usable for everyone else as well is, of course, a wonderful goal and something that many of us care a lot about. But I think it's important not to lose sight of the fact that world-wide adoption on the order of Windows is not a requirement for the Debian project to be a success.

Debian is successful every time I boot a system and it's running Debian, every time Debian solves my problems, every time I can fix something I ran into because it's Debian and I can help make it better. It's fun if I can get more people to use Debian, and it's important to have an influx of new blood and new ideas to keep Debian fresh and responsive, but that's about keeping Debian successful, not about making Debian successful.

If we have enough developers to maintain and improve Debian even at the rate that we're maintaining and improving Debian today, to me that's a success, and I don't really care whether the percentage of Debian users in the broader computing context ever moves off of 0.02%. One of the great things about free software is that we're not a business: we don't live or die by market share, we aren't going to get bought out by someone else if we don't become a big enough fish, and we don't have to grow 10% a year or implode. It would certainly be nice to attract more people and more users and improve even faster, and I certainly wouldn't want to stand in the way of that, but it's not part of my metric of success.

2012-01-11: krb5-sync 2.2

The big news is that, as of this release and thanks to work by Sam Hartman, using this password and status synchronization toolkit no longer requires patching the Kerberos implementation if you're using MIT Kerberos 1.9 or later. The name of the module has also now changed to krb5_sync.so and is installed in a directory inspired by the MIT Kerberos plugin directory. A patch is still needed if you use Heimdal as your KDC implementation.

Support for -randkey password changes (by just ignoring them) has been added in this release, thanks to Dominic Hargreaves.

For the tools, krb5-sync-backend's password command now accepts the password on standard input, which will work better when used as a remctl backend and won't expose the password to other local system users. krb5-sync now reports a proper error message instead of segfaulting if the local configuration is not complete.

There are also other, more minor build system and portability fixes.

You can get the latest version from the krb5-sync distribution page.

2012-01-12: git-pbuilder 1.27

Two changes in this release. First, Guido Günther submitted a patch (which I then mangled) to notice if the DIST ends in -backports and git-pbuilder is calling the builder with an action like create and, in that case, passing --othermirror with the Debian backports mirror. This lets one create new build chroots for squeeze-backports with just:

    env DIST=squeeze-backports git-pbuilder create

without any additional configuration required.

Second, Rafał Długołęcki pointed out the cause of the weird error message that I'd been noticing for years: "W: /root/.pbuilderrc does not exist". git-pbuilder runs the builder with sudo when performing an action like login or create, but sudo resets the environment including HOME, so it tries to load .pbuilderrc from root's home directory. This isn't very useful, so now git-pbuilder checks if a .pbuilderrc exists in the user's home directory and loads it with an explicit --configfile option if it does.

You can get the latest version from my scripts distribution page, and I expect it will turn up in the git-buildpackage package before long.

2012-01-25: Do symbols files make sense for C++?

I also posted this to debian-devel, which is where the discussion will be, but I wanted to post it on my journal as well so that others who may be interested will see it on Planet Debian. For those not familiar with Debian, this concerns the implications of a new feature for how Debian tracks shared library dependencies.

I'm currently working on the Policy modification to document (and recommend) use of symbols instead of shlibs, but I'd only personally used symbols with C libraries. Today I decided that I should try adding a symbols file to a C++ library, particularly if I'm going to recommend everyone do it. I tried this exercise with xml-security-c, which is, I think, a reasonably typical C++ library. Not the sort of core C++ library that would sit at the center of the distribution, but a random software package that's in Debian because other things use it.

The experience was rather interesting, and I ended up uploading the new version without a symbols file and continuing to just use shlibs. That's for the following reasons:

  1. The generated symbols file was HUGE. Hundreds of lines. This is a marked difference from the typical C symbols file, which is of quite manageable size. Some of that is that the library provides a lot of different classes, but some of it is that C++ just generates a lot of exported symbols. There's no way that I could do what I would do with a C library and understand those symbols, why they're there, and whether they are likely to have changed between revisions.

  2. Generating a reasonable symbols file was a pain. Generating an unreasonable symbols file that just contains all of the mangled symbols is largely mechanical and uninteresting, but that symbols file doesn't seem to me to convey useful information. So I did some scripting to translate the symbols back with c++filt, and add (c++) tags, and then try to understand what I was looking at and figure out whether I should sort the symbols list because the default sort is by mangled name, which is meaningless. This is a rather unappealing process. It's not particularly difficult, but it's very awkward and feels like it's missing vital tools.

  3. The resulting symbols file is incomprehensible to someone without strong knowledge of C++. It's full of opaque entries that don't make sense to the non-C++ programmer, which I suspect is a substantial number of people who package C++ libraries for Debian. I know enough C++ from school that I can evaluate security fixes, make simple patches, and review upstream changes, and I think that's all that should be needed to package things for Debian. But I'm deeply uncomfortable producing a symbols file on my own that contains entries for things that I know nothing about and cannot evaluate when they've last changed, like "non-virtual thunk to FooClass::~FooClass@Base".

  4. Once I had a symbols file that resulted in a successful build and that I could have uploaded, I started thinking about how I was going to maintain it. With a C program, I would change the symbols file versions when the underlying function implementation changes in a way that may not offer eqiuvalence, similar to bumping shlibs. I realized that I was going to have no idea when that happened, and the only way that I would maintain the symbols file would be to either trust upstream to maintain ABI equivalence and therefore only change the symbols file when upstream changes the SONAME, or not trust upstream to maintain ABI equivalence and therefore change all the versions with each new upstream release. That gives me exactly the same semantics as a shlibs file, so what's the point in having a symbols file?

  5. The exported symbols of the library contained many symbols that obviously weren't really from that library, but instead were artifacts of the C++ compilation process, things like instantiations of std::vector. Do those go into the symbols file? Do they change from architecture to architecture? If they disappear again, is that actually an ABI break? How do I know? It's all very mysterious, and while shlibs provides the same semantics as just ignoring this, at least I'm not then including in the package data, generated by me, things that I'm just blindly ignoring.

I came away from this experience thinking that I should revise the Policy amendment to say that symbols files are really for C libraries and for C++ libraries with either a tightly maintained symbol export list or maintained by a C++ expert, and that most C++ library maintainers should just not bother with this and use shlibs, bumping the shlibs version or not based on their impression of how good upstream is at maintaining ABI equivalence.

But that feels like a result contrary to what I had previously thought was the intended direction, so I wanted to ask the Debian development community as a whole: am I missing something? Are these symbols files actually useful? Am I missing some trick to make them useful?

2012-01-27: Summary of C++ symbols experience

This is a follow-up to my previous post about the way that Debian's library symbols analysis support interacts with C++ libraries, based on further debian-devel discussion. The primary outcome of that discussion was to point me at the pkg-kde-tools utilities for handling maintenance of these files, which solved a bunch of my problems.

As before, this was also posted on debian-devel, which is where I expect most of the discussion to be.

First of all, for those who haven't explored the pkg-kde-tools infrastructure for this, it looks like the effective process goes something like this:

  1. Do a local build of the package and then generate an initial symbols template for that architecture. So, for example, I did:

        fakeroot debian/rules build install
        pkgkde-gensymbols -plibxml-security-c16 -v1.6.1 -Osymbols.i386 \
            -edebian/libxml-security-c16/usr/lib/*/libxml-security-c.so.*.*
        pkgkde-symbolshelper create -o debian/libxml-security-c16.symbols \
            -v 1.6.1 symbols.i386
    

    This generates an initial symbols file that will work for i386.

  2. Build and upload this version of the package. Now wait for all the buildds to fail (because they will, on probably nearly every other architecture than your local one).

  3. Download all of the build logs and feed them back into pkgkde-symbolshelper so that it can adjust your symbols template for all of the architecture variations. So, for example, I did:

        pkgkde-getbuildlogs
        pkgkde-symbolshelper batchpatch -v 1.6.1 *_unstable_logs/*.build
    

    This will generate rather nice annotated symbols files with appropriate arch lists and using subst tags where appropriately (such as when an argument to a function is a size_t).

  4. Modify your package to use pkg-kde-tools during the build, because subst tags are a really nice way to handle a lot of C++ symbol patterns but #533916 against dpkg-dev is wontfix, so you need the pkg-kde-tools replacement for dpkg-gensymbols. I added a dependency on pkg-kde-tools and then added the (undocumented) pkgkde_symbolshelper add-on to my dh --with flag.

  5. Build and upload the package again. Now it will hopefully build on all architectures.

This is, as you might expect, a fair bit of work, but it does work, and what comes out the other end is a symbols file that works across the current buildd architectures and detects any mistakes by upstream that change the symbols exported.

However, it does have some problems, some obvious, and some more subtle. Here's a list of the issues that I see:

  1. It feels like this symbols file is still likely to be fragile. While pkg-kde-tools detects a lot of template instances and marks them optional, I was still seeing a lot of "leaked" symbols from other C++ libraries that my package build-depends on, and which I suspect may also disappear. There's also the problem of optimizers eliminating some things that are inlined; for example, that appears to be the variation that my package sees on arm. And for a substantial C++ library, the symbols file is HUGE. Over 12,000 lines for one of my libraries. It's kind of scary to think of how many fragility landmines are buried under there.

  2. By the nature of this process, it's very sensitive to the current supported architectures. I expect any package with a symbols template built this way stands a good chance of FTBFS on a new architecture. For example, my symbols file has a lot of !armel !armhf patterns, which don't appear to be anything unique to arm except that the C++ compiler behaves slightly differently there due to inlining, and those are the only two architectures in the current set with that pattern.

  3. The lack of dpkg-gensymbols support for subst is very unfortunate, since it's used by the pkg-kde-tools utilities and it's the Right Thing To Do for symbols that take, for example, size_t as an argument. I would strongly encourage the dpkg maintainers to accept the work in #533916 except maybe for the vt work, since the alternative is to generate regexes that are significantly worse at capturing mistakes (since they're going to accept either int or long, rather than only whatever one corresponds to size_t, for example), or to embed a lot of specific architecture restrictions listing, for instance, all 64-bit architectures and guarantee FTBFS for every new architecture.

  4. It's a little hard on the infrastructure, since adding a symbols file requires uploading a package that's guaranteed to fail to build almost everywhere just so that you can get all the build logs to feed into the machinery. I always feel guilty about doing that.

  5. It's still not clear that the benefit is worth the amount of effort, since I expect most C++ libraries to require frequent SONAME changes anyway, which means that the long-term binary compatibility angle of symbols is probably futile. Mostly, all of this is to give you a tool to do ABI checking, which will catch some mistakes but not all of them. And I'm guessing it's going to be substantially more work than maintaining symbols for a C library if one actually does it properly and checks every missing symbol and change in new builds.

I'm going to try this for the Shibboleth packages for a while since I want to see what it's like across a new upstream release, but I'm still very much not convinced that this is a useful use of a packager's time.

2012-01-28: First 2012 haul

It had been a while since I placed a book order, and since I'm doing quite well at reading regularly so far this year, I decided to treat myself.

Lauren Beukes — Zoo City (sff)
Mark Bittner — Wild Parrots of Telegraph Hill (nonfiction)
Jacqueline Carey — Saints Astray (sff)
Edward Chancellor — Devil Take the Hindmost (nonfiction)
C.J. Cherryh — Regenesis (sff)
Emma Donoghue — Landing (romance)
Doris Egan — The Gate of Ivory (sff)
John Maynard Keynes — Essays in Persuasion (nonfiction)
Scott Lynch — Red Seas Under Red Skies (sff)
Sarah Monette — The Bone Key (sff)
Joanna Russ — How to Suppress Women's Writing (nonfiction)
Barry Schwartz — The Paradox of Choice (nonfiction)
Rory Stewart & Gerald Knaus — Can Intervention Work? (nonfiction)

Last week was pretty much a total loss from an organization and schedule standpoint, but I'm recovering this weekend and planning on getting back to a regular schedule for the next week. When I was holding with that, I was doing quite well at both reading and exercise, so time to restore that pattern.

Last modified and spun 2014-11-07