Posts for January 2014

2014-01-01: 2013 Book Reading in Review

What a strange year.

2013 was marked by a whole sequence of entirely unexpected events, including multiple major work upheavals. For large chunks of the year, I had very little time or emotional energy for personal reading goals, and particularly for writing reviews. I declared personal amnesty on most of my intentions halfway through the year, and all the totals will reflect that. On the plus side (although not for reading and reviews), it was a great year for video games.

Next year, there will be no specific goals. Between continuing work fallout, a very busy project schedule, my intent to keep playing a lot of video games, and various other personal goals I want to take on, I'm going to take the pressure off of reading. Things will be read and reviews will be written (and I'm going to make more of an effort to write reviews shortly after reading books), but I'm not going to worry about how many.

The below statistics are confined to the books I reviewed in 2013. I read six more books that I've not yet reviewed, due to the chaos at the end of the year. Those will be counted in 2014.

There were no 10 out of 10 books this year, partly due to the much lower reading totals and partly due to my tendency this year to turn to safe comfort reading, which is reliably good but unlikely to be exceptional. There were, however, several near-misses that were worth calling out.

My favorite book of the year was Neal Stephenson's Anathem, which narrowly missed a 10 for me due to some fundamental problems with the plot premise. But this is still an excellent book: the best novel about the practice of science and philosophy that I've ever read. Also deserving mention are K.E. Lane's And Playing the Role of Herself, lovely and intelligent lesbian romance that's likely to appeal even to people who would not normally try that genre, and Guy Gavriel Kay's River of Stars. The latter isn't quite at the level of Kay's earlier Under Heaven, but it's still an excellent work of alternate historical fiction in a memorable setting.

A special honorable mention goes to Lisa O'Donnell's The Death of Bees. It requires a lot of warnings for very dark subject matter and a rather abrupt ending, but it's been a long time since I've cared that much about the characters of a book.

My favorite non-fiction book of the year was Gary J. Hudson's They Had to Go Out, a meticulously researched account of a tragic Coast Guard mission. The writing is choppy, the editing could have been better, and it's clear that the author is not a professional writer, but it's the sort of detailed non-fiction account that can only be written by someone who's been there and lived through similar experiences. Also worth mentioning is Mark Jason Dominus's Higher Order Perl, which was the best technical book I read all year and which I found quite inspiring for my own programming.

The full analysis includes some additional personal reading statistics, probably only of interest to me.

2014-01-02: A quick copy-paste UNIX trick

I ran across this trick a long time ago and now use it regularly, but it occurred to me today that I don't remember seeing anyone else mention it. So, just in case it helps someone else:

I often find myself wanting to copy and paste the contents of several small files into an email message, particularly when explaining Debian packaging. I want to show all the file contents with little headers showing which lines come from each file. This gets tedious if I actually do this file-by-file. The obvious approaches, from including them into my editor to cat'ing them one-by-one and pasting them, are tedious and require writing the headers by hand. Obviously, cat doesn't help, since it obliterates the difference between the files.

What does help, however, is good old head (or tail, for that matter). People normally use those utilities with standard input or a single file, but you can give them multiple files, and they do just about exactly what you want. Consider this shell transcript:

    % head -n 20 *.install
    ==> liblog4shib-dev.install <==
    debian/log4shib-config.1 usr/share/man/man1

    ==> liblog4shib-doc.install <==
    usr/share/doc/log4shib-*/api/html usr/share/doc/liblog4shib-doc

    ==> liblog4shib1.install <==

Just pick some value for -n that's longer than any of the files (I normally want to show the contents of small files anyway), run head on all the files whose contents you want to show, and then you can cut and paste its full output directly into your email message.

Bonus: This behavior is even guaranteed by POSIX (something that I didn't realize until I checked while writing this post).

2014-01-03: lbcd 3.5.1

This is a small bug-fix release of this load-balancing system status daemon. The primary purpose of the release is to merge the various fixes to the new systemd unit files that I made in the Debian package after a broader code review. This release also fixes a long-lived oversight and sets lbcd's listening sockets to close-on-exec so that they're not inherited by external weight commands (already handled under systemd by socket activation) and improves the lbcd man page.

You can get the latest version from the lbcd distribution page.

2014-01-04: More on displaying files with head

That was fun! Since my previous entry on using head to display the contents of several files in a form that's easy to cut and paste, multiple people have sent elaborations or related tricks. It seemed like it would be a good idea to post a roundup, since I learned a bunch.

Multiple people (I think Josh Triplett was the first) pointed out that one can avoid having to pick a sufficiently large value of -n by instead using:

    head -n -0 *.install

With GNU head at least, a negative number says to print out all lines of the file except that many at the end, so -0 displays the whole file, regardless of size. Unfortunately, while this works anywhere that I am likely to run it, it's not specified by POSIX, while the original is.

Another variation, pointed out by Buck Huppmann, is:

    tail -n +0 *.install

The +0 syntax is required by POSIX, unlike the -0 syntax for head... but unfortunately POSIX doesn't require that tail supports multiple files and the headers, although it does for head.

Buck also pointed out that including the -v flag will always force the header even if there's only one file, which is useful. (Although be warned that -v isn't a POSIX-recognized flag.)

Markus Raab also pointed out the xsel utility, which I'd heard of but hadn't ever used. If the goal is to cut and paste the output, using:

    head -v -n -0 *.install | xsel -i

avoids the cut part by dumping the result directly into the X selection. Buck pointed out xclip, which does the same thing. Both can be used with the -o flag inside an editor to paste as well if you don't want to reach for a mouse. In vi :r !xclip -o, and in Emacs, C-u M-! xclip -o.

Finally, Guillem Jover metioned that:

    grep . *.install

does sort of the same thing with a different output format that may be more useful depending on what you're doing. (I find it less human-readable but more machine-parsable.)

2014-01-05: rra-c-util 5.1

The primary addition in this release is the Autoconf macros and header wrapper that I developed for lbcd to add systemd support. These are inspired by the code in systemd's daemon(7) man page but adjusted for my normal software conventions. This is just preliminary and doesn't yet stub out all of the functions, only the ones I've needed so far.

Also in this release, I fixed a bug with process_start_fakeroot, and hence with remctld_start_fakeroot, introduced in the last release when I rewrote the TAP process management add-on.

You can get the latest release from the rra-c-util distribution page.

2014-01-06: remctl 3.7

remctl is the middleware layer that we use everywhere at Stanford. It's a simple GSS-API-authenticated network service that supports running commands with ACLs. There are client bindings available for a wide variety of programming languages.

This release fixes a couple of irritating bugs: the client library leaked memory when remctl_set_ccache was used (which was affecting mod_webkdc from WebAuth), and Net::Remctl::Backend didn't validate argument counts correctly when one of the arguments came from standard input (which affected krb5-sync). I also worked around a bug in RHEL 5's Module::Build, and added sanity checking to Net::Remctl and related classes to ensure that the object argument wasn't undef.

Also new in this release are support in the remctld server for systemd startup notification and socket activation, and (via the -Z) flag) support for upstart's expect stop synchronization method.

This is mostly a "clearing the decks" release in advance of more significant work. The next release will replace the server event loop with libevent, in preparation for further improvements in how the server can handle persistent worker children.

You can get the latest release from the remctl distribution page.

2014-01-15: kadmin-remctl 3.6

Another minor tweak to the remctl backend for Kerberos administrative functions that we use at Stanford. This still isn't the long-awaited rewrite that might make this more generally useful outside of Stanford's specific requirements.

This release adds support for setting a password expiration on newly-created accounts via a configuration setting in the local configuration file. The password expiration period can be configured per-instance. (Eventually, this should be handled via policy support instead, but this was easier for my immediate purposes.)

This release also maps password quality error messages on create or reset_passwd to generic messages in the Heimdal backend. This is only of interest to sites such as ours that have patched Heimdal to do password quality checks for administrative actions as well as user-initiated password changes. The kadmin protocol unfortunately doesn't have a way to pass back the verbose password quality error.

You can get the latest version from the kadmin-remctl distribution page.

2014-01-28: C TAP Harness 3.0

This is my test harness and C and shell library for writing test suites using the Test Anything Protocol (also used by Perl). The changes in this release aren't that major, but I changed one API in a non-backward-compatible way, so I bumped the major revision.

That change is to test_cleanup_register, whose callback functions now take two arguments instead of one. The first, as before, indicates whether the test case succeeded or failed. The second is now true if the callback is running in the primary process (the one in which plan or plan_lazy was called), and false otherwise. Callback functions are now called for all processes, not just the primary one. This allows cleanup of per-process resources, like memory, in the child processes, which allows for easier valgrind analysis.

Also in this release, the test harness was modified to reopen standard input to /dev/null when running a set of tests from a list, rather than inheriting standard input from the test driver. The test harness also now cleans up all extraneous copies of open file descriptors used during the setup of standard input, output, and error to avoid leaking harmless but noticeable file descriptors to tests.

You can get the latest release from the C TAP Harness distribution page.

2014-01-28: rra-c-util 5.2

Another release of my general collection of utility libraries, Autoconf probes, general test cases, and similar things.

The major change in this release is the addition of Autoconf probes and portability wrappers for libevent. Now included are Autoconf macros to probe for libevent (mandatory or optional) either via pkg-config or not, following the pattern of other probes in rra-c-util. Also included is a portability wrapper, probes, and some additional glue code that allows programs to be written against the libevent 2.x API and work (mostly) with the libevent 1.4 API, albeit with some missing functionality.

This release fixes the systemd unit directory probe to not produce shell errors if pkg-config isn't available on the local system. It also fixes a syntax error when building portable/krb5.h with a C++ compiler.

There are multiple improvements to the Kerberos and process TAP add-ons: memory is properly freed in all processes, making exhaustive memory allocation analysis easier; process output is flushed after stopping the process instead of before; the method used to stop processes is cleaner; a memory leak when running via fakeroot has been fixed; and full use is made of the new test_cleanup_register API in C TAP Harness.

This release also adds a new message_handlers_reset function that restores all message handlers to their defaults and frees any memory allocated to set different handlers. This again is mostly useful when doing exhaustive memory allocation testing.

You can get the latest release from the rra-c-util distribution page.

2014-01-28: remctl 3.8

remctl is the middleware system that we use everywhere at Stanford. It's a very simple way to run commands, constrained by ACLs, over a GSS-API-authenticated with good encryption.

This release doesn't have a lot of obvious user-visible changes. Rather, it's an internal refactoring and cleanup release that will make further changes easier.

The major change is that the process event loop, which runs the command and sends its output back over the wire to the client, now uses libevent. This makes the code cleaner and fixes various problems previously caused by portability, such as the lack of proper handling of SIGCHLD forcing a short poll interval to see if the child had exited.

While working on that, I also found and cleaned up several other minor issues: the maximum output per remctl token now matches the protocol maximum, rather than the arbitrary and strange choice of 65,000 octets, and I fixed a few minor memory leaks in the server and client that no one probably would have noticed. The server also now writes its PID file atomically.

You can get the latest release from the remctl distribution page.

Last modified and spun 2017-03-25