Posts for March 2013

2013-03-10: Small non-fiction haul

Long time no write. A variety of things, including getting sick with a particularly bad cold (I'm still coughing up phlegm), led to my schedule and general life organization getting turned on its head, and I'm only slowly recovering. Work gets top priority (including a security release of OpenAFS), so I'm afraid non-work things such as reviews and journal posts (and Debian, and non-work-related software development, and conversations with friends...) have been getting second shrift. Hopefully this is finally starting to improve, and I have a block of time for working remotely coming up that should help.

In the meantime, more books have, of course, been acquired. This particular order was mostly to get one book (The Making of the Indebted Man), which I'm going to read and discuss as part of a surprisingly fun political discussion about basic income guarantee and, somewhat by extension, Marxist analysis of capitalism. But it's impossible to buy just one book, similar to how it's impossible to eat just one potato chip:

Neil Barofsky — Bailout (non-fiction)
E. Gabriella Coleman — Coding Freedom (non-fiction)
Joshua Foer — Moonwalking with Einstein (non-fiction)
Maurizio Lazzarato — The Making of the Indebted Man (non-fiction)
Isabel Wilkerson — The Warmth of Other Suns (non-fiction)

I of course preordered Coding Freedom and am quite looking forward to reading it, although it will surprise no one to hear that I have rather a long queue. The Warmth of Other Suns I picked up based on a recommendation by Ta-Nehisi Coates, whose writing for The Atlantic I wholeheartedly recommend.

I have such a large pile of books that I've read but not yet reviewed that I cringe to think about it. But no progress will be made on that this evening (damn you, Benjamin Franklin! *shakes fist*). More tomorrow if I survive five and a half hours of scheduled forced social interaction.

2013-03-12: WebAuth 4.4.3

WebAuth is the site-wide web authentication system that we use at Stanford. After a lot of time focusing on things at work other than coding, it's been my primary job for the past nine months or so, which has been a lovely change.

I wasn't planning on doing another 4.4 release and instead focusing on 4.5.0 (which is well underway and adds significant new features to multifactor support), but I kept finding bugs, including several that were quite embarassing. So this is another bug-fix release, and hopefully the last one before 4.5.0.

WebAuthTrustAuthzIdentity enabled WebAuthDoLogout in the same scope (ever since it was added in 4.4.0). It's horribly embarassing to have been writing C for more than 20 years and still manage to omit a break in a switch statement.

Benjamin Coddington found another bug in authorization identity handling where if the user changed their authorization identity to match their authentication identity, their authentication would be rejected. We now detect that case and just discard the authorization identity if it's the same as the authentication identity.

There are multiple fixes in the mod_webauth logging of bad app tokens: expired app tokens are now logged with a proper message (and at info level rather than error), empty app tokens (created internally and therefore seen by subqueries before fixup to delete expired app tokens) are now just ignored, and invalid app tokens don't result in spewing binary strings into the Apache error log.

mod_webauthldap has, since the beginning, refused to add more than 127 values of a multivalued LDAP attribute to the environment. We ran into that limit with entitlements, so it's now been removed. This runs the risk of overflowing the environnment, but I did some calculations, and it looks like you're going to have to try really hard and have impressively large multivalued attributes to create that problem. If it actually becomes a problem for someone, I'll add a configuration option.

There's also a syntax fix to the default WebLogin error template and a variety of minor bug fixes, mostly around error handling conditions, to correct problems caught by clang --analyze. (The current master branch gets a completely clean bill of health from clang --analyze using clang 3.0.)

You can get the latest release from the official WebAuth distribution site or from my WebAuth distribution pages.

2013-03-14: C TAP Harness 2.0

This package is my pure C test harness that understands the TAP protocol. It's akin to prove, except written in C. It also provides a TAP library for C and shell scripts, which is useful even if you don't want to use the harness.

I was going to just do a small release with no major changes, just some cleanup and merging of changes from elsewhere, but I do hate making tiny releases and I'd been meaning to get back to doing some larger development for a while. So this release has several major changes, including one backward-incompatible one.

The backward-incompatible change is that, to specify a list of tests to run, one now needs to use the -l option to runtests. That's because the default interpretation of the command line is now that it is a list of tests to run and summarize. This is more consistent with other test harnesses, such as prove. Also in this release, runtests will look for the test name as given if it can't find it by appending -t or .t, which allows one to pass the actual name of the test executable on the command line (with or without the -o option). This makes command-line completion less annoying.

There are a few other minor fixes to display: output is flushed after each test executable completes even if the output isn't a terminal, which produces better display when teeing to a log file, and partial status of tests that use lazy plans now show the total test count as "?" to match the behavior of prove. I also fixed a few errors when displaying a test abort to a terminal.

You can get the latest version from the C TAP Harness distribution page.

2013-03-15: C TAP Harness 2.1

I probably should have updated more of my other packages before kicking out the 2.0 release, but I wanted to get something out last night.

The change to try to find test executables without any extension as well as with the -t or .t extensions broke backward compatibility. In some of my test suites, I use a helper program with no extension that's run by the actual test program with a -t extension. If the helper program is compiled and hence in the build directory, and the actual test is a shell script and hence in the source directory, then the new runtests would find and try to run the wrong program.

Fix this problem by instead searching all possible locations for the test program with a particular suffix before moving on to try the next suffix on the list. This should provide the same functionality while being backward-compatible.

You can get the latest release from the C TAP Harness distribution page.

2013-03-15: rra-c-util 4.8

This package is my collection of random utility functions, sort of like my version of Gnulib. (At some point, I do want to retire the bits that just duplicate Gnulib, which is less and less of it, and figure out how to properly use Gnulib for my packages as well.)

The big changes in this package put the name somewhat to lie: I finally took the time to write some comprehensive infrastructure for integrating Perl tests. This is primarily in the form of three new modules and a bunch of tests that use them.

The modules are Test::RRA, which provides some general facilities for handling maintainer-only tests and skipping tests when prerequisites are missing; Test::RRA::Config, which reads configuration from a separate file and allows me to use the same code verbatim in multiple packges; and Test::RRA::Automake, which provides a bunch of useful functions for intergrating tests written in Perl into packages that use Automake.

Some of the test scripts were in the previous version, but they've now been overhauled to use the new modules, which makes them much simpler and more robust. But I also added a new perl/t directory, which holds all the test scripts that I use for Perl module testing, either standalone or embedded in a larger package.

There are a variety of other changes, but they're minor or internal. It is worth noting, though, that all the code in rra-c-util now passes clang --analyze with clang 3.0.

You can get the latest version from the rra-c-util distribution page.

2013-03-15: On (not) drinking

Captain Awkward, which is still the best advice column on the Internet, just posted an entry about someone being pestered to drink after saying that they don't drink. As another one of those people who just don't drink, but not for any specific reason that people seem to accept, I felt like blathering a bit more about that. This is mostly about me rather than about the thread over there, so I figured I'd do it on my own journal.

First, I completely agree with the response that it's no one else's business why you do or don't drink, and people who persist in trying to get you to drink (or do anything else, no matter how innocuous) after you have clearly told them no are being rude and have forfeited the right to further conversation. But it also really frustrates me to see that people have to cope with that situation, particularly since I suspect it's gendered. My personal experience (as a man, and one who has a fairly firm way of expressing myself naturally) is that I say I don't drink and that's the end of it. Occasionally someone will ask me why. I've never had anyone pester me beyond that. I think the age of the letter writer (22) has something to do with this, but I suspect gender sadly also has something to do with this. People seem to be much more willing to tell women what to do and not take no for an answer.

It's unbelievably rude when people do that. Don't do that. If you decide you can pester someone about their personal choices because of their gender (and, for that matter, their age, or any other reason), you deserve to get your head handed to you on a plate.

Second, I wonder if some of the reason why I don't run into problems with this is that I work on a college campus. The latter means that there's a lot of surrounding culture around drinking and dealing with drinking, such as the university policy that any party have EANABs (Equally Attractive Non-Alcoholic Beverages) available as well. There's a bit of a cultural understanding that pushing people into drinking isn't okay.

Third, since people do occasionally ask why I don't drink, or assume that anyone who doesn't drink is doing so for religious reasons or because they're an alcoholic, or might object to them drinking, here are a few reasons why I don't drink. It may be useful information in case you run into more people in the world like me.

So, that's me. It makes no difference at all to me if other people drink. Enjoy! You won't offend me, there isn't any overtone to it, I'm not being judgemental when I say I don't drink, and it's not a dangerous topic or anything. There are probably others like me. If you run into us, all you have to say after "no thanks, I don't drink" is "oh, okay."

2013-03-25: kadmin-remctl 3.3

kadmin-remctl is a remctl wrapper around the kadmin protocol that we use at Stanford to provide an easier API to Kerberos administration (for Java applications in particular, but also somewhat for humans) and to use a finer-grained ACL than Kerberos kadmin provides.

This package is still kind of a mess, with some Stanford-specific bits and a whole lot of messy Perl code, but we had some serious bugs that I needed to deal with. So this is mostly a bug-fix release without the much-delayed cleanup. (More on that in a moment.)

The primary problem this release addresses is a very annoying network problem that we've been having at Stanford. Some interaction between our new OpenFlow fabric and our current firewalls causes some number of TCP connections to just be dropped under heavy load. Since we do a lot of kadmin queries (although we're working on various caching strategies to cut down on that), kadmin sees this a lot, which results in failures that cause user-noticable problems in various applications. After some experimentation, though, we found that most of the failures were on the original connection, so catching connection failure and retrying works around most of the problem (at the cost of a delay).

That was only the beginning of the odyssey, however. When I added the retry code, I found that we were improperly configuring Heimdal::Kadm5 to actually throw exceptions. And then, even after fixing that, I found that error messages were still printed to standard error. It turns out that the underlying Heimdal libraries print warnings to standard error by default unless the application directs the log messages somewhere else, and Heimdal::Kadm5 doesn't have a look available to do that. So I ended up adding a bunch of ugly code to close standard error while doing the initial connection.

This release also cleans up a bunch of the error handling, including going back to the behavior that the code always should have had: check_passwd returns a non-zero status if the password is rejected. I lost an argument with a co-worker about that years ago, but they've since left and I still think that's the right thing to do. The new code also uses IPC::Run to run commands, which makes for much shorter and saner code, and improves error reporting when trying to change the password of a disabled account with Heimdal's kpasswd. It also strips whitespace from the username in passwd_change.

At some point, I'm going to rewrite this software completely, but what that really needs is a much saner underlying Kerberos module. One of the problems with all existing Kerberos Perl modules is that they all use various hacks to try to build with different Kerberos libraries within the ExtUtils::MakeMaker framework, which is totally inadequate. (That's also why there aren't any Perl modules linked with the server version of the kadmin libraries, so they're all painfully slow and require a keytab even when running directly on the KDC.)

I think the answer is a brand new Perl module that incorporates my portability layer on top of Kerberos libraries and a configure script that does proper probing for Kerberos library functionality, and then integrates that Autoconf probing into a Module::Build infrastructure. There are some bits on CPAN that one could start with, but not all the proper glue, so this is a bit of a project. I also have to figure out how to link different extensions in the project with different libraries, since having separate distributions for the server and client kadmin modules (which share 95% of their code) is dumb, but they have to be separately linked with the appropriate library.

Sometime, when I have some time, I'll do that, and then rebuild a remctl interface on top of that infrastructure with a much better protocol.

In the meantime, you can get the latest version of kadmin-remctl from the kadmin-remctl distribution page.

2013-03-26: remctl 3.4

remctl is the client/server protocol that we use for basically all our middleware connections at Stanford. It uses Kerberos GSS-API for authentication and privacy and is designed to be as simple as possible, just running a command on the server with the provided arguments. We've found this design to be extremely effective and a significant improvement over trying to put web servers everywhere to use something like REST.

The primary feature in this release is a new Net::Remctl::Backend Perl module that automates a lot of the work of creating remctl backend scripts written in Perl. Over the years, we've developed a lot of conventions for things like help output and command-line processing, and this rolls those conventions into a module that handles a lot of the work of formatting help output and handling command dispatch. This is just the first cut, but it already supports argument validation, handling standard input, formatting help output, and handling per-option command-line flags. Later versions will feature better integration with Kerberos and hopefully better integration into the new remctl help and summary features.

Also in this release are new C APIs contributed by Jeffrey Hutzelman that allow one to start a remctl connection over an existing sockaddr, a list of struct addrinfo results, or an open socket.

In the bug fix department, this release removes all the prototypes from Net::Remctl functions following current Perl best practices (prototypes cause weird context issues and behave in surprising ways), and rejects the empty command to remctl_command rather than trying to malloc 0 bytes.

You can get the latest version from the remctl distribution page.

2013-03-27: wallet 1.0

Finally.

The wallet is a system management tool for storing and retrieving secure data. I originally wrote it as part of our Kerberos v5 migration project to replace the old Kerberos v4 system we had for distributing srvtabs. It's still mostly used at Stanford for distributing keytabs, but it's extensible and supports storing arbitrary secure data. We also use it internally for storing database passwords, SSL private keys, and anything else that needs to be kept secure and retrieved later. It supports a rich ACL mechanism and namespace enforcement and is built on top of remctl for security and confidentiality.

I did a bunch of development on it originally in 2006 and 2007, but then it solved our immediate problems and I didn't have much time to work on it. I used pre-1.0 versions because there was so much more on it that I wanted to do. But we've been running it in production for years, and while there's still tons more work that needs to be done, it really deserves a 1.0 version. Jon Robertson converted the database backend to DBIx::Class (which added PostreSQL support) and implemented schema upgrading, which were two of the major things that were keeping me from putting a 1.0 version on it. So it's time.

Other new features since the previous release include a change to the default ACLs so that owners of objects can destroy them by default, a new ldap-attr ACL type that checks whether the caller has a particular attribute set in LDAP (which will let us use our entitlement system to control access), and support for storing and managing key rotation for WebAuth keyrings. There's also a new acl check command to determine whether an ACL exists and a new comment field and command to store an arbitrary comment about an object.

Our local naming policy has also gotten much more complex, so I turned it into a module to provide an example to others of how to implement some reasonably detailed namespace and object autocreation controls.

You can get the latest version from the wallet distribution page. I'm going to upgrade Debian packages to experimental or unstable (depending on the freeze status) as soon as I get a chance to write a bit more documentation and hopefully figure out debconf integration with the database setup support inside Debian for the server piece.

2013-03-28: lbcd 3.4.1

lbcd is an agent that we run on systems that use software load balancing. It returns a weight and an increment that's used by the lbnamed DNS server.

This is a minor feature release. The main point of this release is to add support for an /etc/nolbcd file (or /usr/local/etc/nolbcd in the default build) that's akin to /etc/nologin except it just drops the system from an lbnamed load balance pool rather than denying all logins. Sometimes it's more convenient to do that than to stop the lbcd daemon, particularly when the latter is monitored by configuration management software.

Also in this release, the lbcd -t mode, which prints all the data to standard output, is now documented and has been improved, and lbcdclient no longer rewrites maximum weights to -1.

You can get the latest release from the lbcd distribution page.

Last modified and spun 2017-03-25