2013-03-10: Small non-fiction haul
Long time no write. A variety of things, including getting sick with a
particularly bad cold (I'm still coughing up phlegm), led to my schedule
and general life organization getting turned on its head, and I'm only
slowly recovering. Work gets top priority (including a security release
of OpenAFS), so I'm afraid non-work things such as reviews and journal
posts (and Debian, and non-work-related software development, and
conversations with friends...) have been getting second shrift. Hopefully
this is finally starting to improve, and I have a block of time for
working remotely coming up that should help.
In the meantime, more books have, of course, been acquired. This
particular order was mostly to get one book (The Making of the
Indebted Man), which I'm going to read and discuss as part of a
surprisingly fun political discussion about
basic income
guarantee and, somewhat by extension, Marxist analysis of capitalism.
But it's impossible to buy just one book, similar to how it's impossible
to eat just one potato chip:
Neil Barofsky — Bailout (non-fiction)
E. Gabriella Coleman — Coding Freedom (non-fiction)
Joshua Foer — Moonwalking with Einstein (non-fiction)
Maurizio Lazzarato — The Making of the Indebted Man
(non-fiction)
Isabel Wilkerson — The Warmth of Other Suns (non-fiction)
I of course preordered Coding Freedom and am quite looking forward
to reading it, although it will surprise no one to hear that I have rather
a long queue. The Warmth of Other Suns I picked up based on a
recommendation by Ta-Nehisi Coates, whose writing for The Atlantic I wholeheartedly
recommend.
I have such a large pile of books that I've read but not yet reviewed that
I cringe to think about it. But no progress will be made on that this
evening (damn you, Benjamin Franklin! *shakes fist*). More tomorrow if I
survive five and a half hours of scheduled forced social interaction.
2013-03-12: WebAuth 4.4.3
WebAuth is the site-wide web authentication system that we use at
Stanford. After a lot of time focusing on things at work other than
coding, it's been my primary job for the past nine months or so, which has
been a lovely change.
I wasn't planning on doing another 4.4 release and instead focusing on
4.5.0 (which is well underway and adds significant new features to
multifactor support), but I kept finding bugs, including several that were
quite embarassing. So this is another bug-fix release, and hopefully the
last one before 4.5.0.
WebAuthTrustAuthzIdentity enabled WebAuthDoLogout in the same scope (ever
since it was added in 4.4.0). It's horribly embarassing to have been
writing C for more than 20 years and still manage to omit a break
in a switch statement.
Benjamin Coddington found another bug in authorization identity handling
where if the user changed their authorization identity to match their
authentication identity, their authentication would be rejected. We now
detect that case and just discard the authorization identity if it's the
same as the authentication identity.
There are multiple fixes in the mod_webauth logging of bad app tokens:
expired app tokens are now logged with a proper message (and at info level
rather than error), empty app tokens (created internally and therefore
seen by subqueries before fixup to delete expired app tokens) are now just
ignored, and invalid app tokens don't result in spewing binary strings
into the Apache error log.
mod_webauthldap has, since the beginning, refused to add more than 127
values of a multivalued LDAP attribute to the environment. We ran into
that limit with entitlements, so it's now been removed. This runs the
risk of overflowing the environnment, but I did some calculations, and it
looks like you're going to have to try really hard and have impressively
large multivalued attributes to create that problem. If it actually
becomes a problem for someone, I'll add a configuration option.
There's also a syntax fix to the default WebLogin error template and a
variety of minor bug fixes, mostly around error handling conditions, to
correct problems caught by clang --analyze. (The current master
branch gets a completely clean bill of health from clang --analyze
using clang 3.0.)
You can get the latest release from the official WebAuth distribution site
or from my WebAuth distribution pages.
2013-03-14: C TAP Harness 2.0
This package is my pure C test harness that understands the TAP protocol.
It's akin to prove, except written in C. It also provides a TAP library
for C and shell scripts, which is useful even if you don't want to use the
harness.
I was going to just do a small release with no major changes, just some
cleanup and merging of changes from elsewhere, but I do hate making tiny
releases and I'd been meaning to get back to doing some larger development
for a while. So this release has several major changes, including one
backward-incompatible one.
The backward-incompatible change is that, to specify a list of tests to
run, one now needs to use the -l option to runtests. That's because the
default interpretation of the command line is now that it is a list of
tests to run and summarize. This is more consistent with other test
harnesses, such as prove. Also in this release, runtests will look for
the test name as given if it can't find it by appending -t or .t, which
allows one to pass the actual name of the test executable on the command
line (with or without the -o option). This makes command-line completion
less annoying.
There are a few other minor fixes to display: output is flushed after each
test executable completes even if the output isn't a terminal, which
produces better display when teeing to a log file, and partial status of
tests that use lazy plans now show the total test count as "?" to match
the behavior of prove. I also fixed a few errors when displaying a test
abort to a terminal.
You can get the latest version from the
C TAP Harness distribution page.
2013-03-15: C TAP Harness 2.1
I probably should have updated more of my other packages before kicking
out the 2.0 release, but I wanted to get something out last night.
The change to try to find test executables without any extension as well
as with the -t or .t extensions broke backward compatibility. In some of
my test suites, I use a helper program with no extension that's run by the
actual test program with a -t extension. If the helper program is
compiled and hence in the build directory, and the actual test is a shell
script and hence in the source directory, then the new runtests would find
and try to run the wrong program.
Fix this problem by instead searching all possible locations for the test
program with a particular suffix before moving on to try the next suffix
on the list. This should provide the same functionality while being
backward-compatible.
You can get the latest release from the
C TAP Harness distribution page.
2013-03-15: rra-c-util 4.8
This package is my collection of random utility functions, sort of like my
version of Gnulib. (At some
point, I do want to retire the bits that just duplicate Gnulib, which is
less and less of it, and figure out how to properly use Gnulib for my
packages as well.)
The big changes in this package put the name somewhat to lie: I finally
took the time to write some comprehensive infrastructure for integrating
Perl tests. This is primarily in the form of three new modules and a
bunch of tests that use them.
The modules are Test::RRA, which provides some general facilities for
handling maintainer-only tests and skipping tests when prerequisites are
missing; Test::RRA::Config, which reads configuration from a separate file
and allows me to use the same code verbatim in multiple packges; and
Test::RRA::Automake, which provides a bunch of useful functions for
intergrating tests written in Perl into packages that use Automake.
Some of the test scripts were in the previous version, but they've now
been overhauled to use the new modules, which makes them much simpler and
more robust. But I also added a new perl/t directory, which holds
all the test scripts that I use for Perl module testing, either standalone
or embedded in a larger package.
There are a variety of other changes, but they're minor or internal. It
is worth noting, though, that all the code in rra-c-util now passes
clang --analyze with clang 3.0.
You can get the latest version from the rra-c-util distribution page.
2013-03-15: On (not) drinking
Captain Awkward, which is still the best advice column on the Internet,
just posted an entry about
someone being pestered to drink after saying that they don't drink. As
another one of those people who just don't drink, but not for any specific
reason that people seem to accept, I felt like blathering a bit more about
that. This is mostly about me rather than about the thread over there, so
I figured I'd do it on my own journal.
First, I completely agree with the response that it's no one else's
business why you do or don't drink, and people who persist in trying to
get you to drink (or do anything else, no matter how innocuous) after you
have clearly told them no are being rude and have forfeited the right to
further conversation. But it also really frustrates me to see that people
have to cope with that situation, particularly since I suspect it's
gendered. My personal experience (as a man, and one who has a fairly firm
way of expressing myself naturally) is that I say I don't drink and that's
the end of it. Occasionally someone will ask me why. I've never
had anyone pester me beyond that. I think the age of the letter writer
(22) has something to do with this, but I suspect gender sadly also has
something to do with this. People seem to be much more willing to tell
women what to do and not take no for an answer.
It's unbelievably rude when people do that. Don't do that. If you decide
you can pester someone about their personal choices because of their
gender (and, for that matter, their age, or any other reason), you deserve
to get your head handed to you on a plate.
Second, I wonder if some of the reason why I don't run into problems with
this is that I work on a college campus. The latter means that there's a
lot of surrounding culture around drinking and dealing with drinking, such
as the university policy that any party have EANABs (Equally Attractive
Non-Alcoholic Beverages) available as well. There's a bit of a cultural
understanding that pushing people into drinking isn't okay.
Third, since people do occasionally ask why I don't drink, or assume that
anyone who doesn't drink is doing so for religious reasons or because
they're an alcoholic, or might object to them drinking, here are a
few reasons why I don't drink. It may be useful information in case you
run into more people in the world like me.
Due mostly to accident, and certainly not due to any specific effort
on my part, I made it to late teenage years without ever having any
significant amount of alcohol. That means I never developed a taste
for it. (I didn't develop an aversion to it either; I just have no
opinion on it. I've had just enough of it to know how it tastes, such
as the occasional socially-indicated glass of wine, but not enough to
form any opinions.) After that point, I figured since I'd not had it
yet, there was no reason to start. Now, some years later, I already
have more hobbies and things I enjoy than I have time and attention
for, and I'm no longer at the point of my life where I'm actively
seeking out new ones just because lots of other people enjoy them.
If alcohol isn't going to be a hobby, and is just something else to
drink, well, alcohol is expensive and generally fairly high-calorie.
In general, I only drink things that have no calories because I like
to drink a lot and I prefer to save my available calories for food
(which I generally enjoy more). Why not save my money and spend the
calories on something I know I enjoy?
I really hate all of the physical symptoms (headache, nausea,
unsteadiness, lack of emotional control, dizziness) one gets from
overdrinking when I've gotten them via other means. I know that one
doesn't necessarily get drunk when one drinks, and one can avoid
getting that drunk even if one gets drunk, but given that I've never
developed any taste for alcohol, why bother to start on something that
I've seen lead to really nasty physical symptoms in lots and lots of
my friends?
I'm remarkably stubborn about things that I do that other people don't
do. Show me cultural pressure contrary to something I'm currently
doing, even if I'm only doing that thing by accident, and I tend to
make it a point of pride for no other reason than I dislike pressure.
(I probably subconsciously exude this, which is probably part of the
reason why people don't argue with me.)
So, that's me. It makes no difference at all to me if other people drink.
Enjoy! You won't offend me, there isn't any overtone to it, I'm not being
judgemental when I say I don't drink, and it's not a dangerous topic or
anything. There are probably others like me. If you run into us, all you
have to say after "no thanks, I don't drink" is "oh, okay."
2013-03-25: kadmin-remctl 3.3
kadmin-remctl is a remctl wrapper around the kadmin protocol that we use
at Stanford to provide an easier API to Kerberos administration (for Java
applications in particular, but also somewhat for humans) and to use a
finer-grained ACL than Kerberos kadmin provides.
This package is still kind of a mess, with some Stanford-specific bits and
a whole lot of messy Perl code, but we had some serious bugs that I needed
to deal with. So this is mostly a bug-fix release without the
much-delayed cleanup. (More on that in a moment.)
The primary problem this release addresses is a very annoying network
problem that we've been having at Stanford. Some interaction between our
new OpenFlow fabric and our current firewalls causes some number of TCP
connections to just be dropped under heavy load. Since we do a lot
of kadmin queries (although we're working on various caching strategies to
cut down on that), kadmin sees this a lot, which results in failures that
cause user-noticable problems in various applications. After some
experimentation, though, we found that most of the failures were on the
original connection, so catching connection failure and retrying works
around most of the problem (at the cost of a delay).
That was only the beginning of the odyssey, however. When I added the
retry code, I found that we were improperly configuring Heimdal::Kadm5 to
actually throw exceptions. And then, even after fixing that, I found that
error messages were still printed to standard error. It turns out that
the underlying Heimdal libraries print warnings to standard error by
default unless the application directs the log messages somewhere else,
and Heimdal::Kadm5 doesn't have a look available to do that. So I ended
up adding a bunch of ugly code to close standard error while doing the
initial connection.
This release also cleans up a bunch of the error handling, including going
back to the behavior that the code always should have had: check_passwd
returns a non-zero status if the password is rejected. I lost an argument
with a co-worker about that years ago, but they've since left and I still
think that's the right thing to do. The new code also uses IPC::Run to
run commands, which makes for much shorter and saner code, and improves
error reporting when trying to change the password of a disabled account
with Heimdal's kpasswd. It also strips whitespace from the username in
passwd_change.
At some point, I'm going to rewrite this software completely, but what
that really needs is a much saner underlying Kerberos module. One of the
problems with all existing Kerberos Perl modules is that they all use
various hacks to try to build with different Kerberos libraries within the
ExtUtils::MakeMaker framework, which is totally inadequate. (That's also
why there aren't any Perl modules linked with the server version of the
kadmin libraries, so they're all painfully slow and require a keytab even
when running directly on the KDC.)
I think the answer is a brand new Perl module that incorporates my
portability layer on top of Kerberos libraries and a configure script that
does proper probing for Kerberos library functionality, and then
integrates that Autoconf probing into a Module::Build infrastructure.
There are some bits on CPAN that one could start with, but not all the
proper glue, so this is a bit of a project. I also have to figure out how
to link different extensions in the project with different libraries,
since having separate distributions for the server and client kadmin
modules (which share 95% of their code) is dumb, but they have to be
separately linked with the appropriate library.
Sometime, when I have some time, I'll do that, and then rebuild a remctl
interface on top of that infrastructure with a much better protocol.
In the meantime, you can get the latest version of kadmin-remctl from the
kadmin-remctl distribution page.
2013-03-26: remctl 3.4
remctl is the client/server protocol that we use for basically all our
middleware connections at Stanford. It uses Kerberos GSS-API for
authentication and privacy and is designed to be as simple as possible,
just running a command on the server with the provided arguments. We've
found this design to be extremely effective and a significant improvement
over trying to put web servers everywhere to use something like REST.
The primary feature in this release is a new Net::Remctl::Backend Perl
module that automates a lot of the work of creating remctl backend scripts
written in Perl. Over the years, we've developed a lot of conventions for
things like help output and command-line processing, and this rolls those
conventions into a module that handles a lot of the work of formatting
help output and handling command dispatch. This is just the first cut,
but it already supports argument validation, handling standard input,
formatting help output, and handling per-option command-line flags. Later
versions will feature better integration with Kerberos and hopefully
better integration into the new remctl help and summary features.
Also in this release are new C APIs contributed by Jeffrey Hutzelman that
allow one to start a remctl connection over an existing sockaddr, a list
of struct addrinfo results, or an open socket.
In the bug fix department, this release removes all the prototypes from
Net::Remctl functions following current Perl best practices (prototypes
cause weird context issues and behave in surprising ways), and rejects the
empty command to remctl_command rather than trying to malloc 0 bytes.
You can get the latest version from the remctl distribution page.
2013-03-27: wallet 1.0
Finally.
The wallet is a system management tool for storing and retrieving secure
data. I originally wrote it as part of our Kerberos v5 migration project
to replace the old Kerberos v4 system we had for distributing srvtabs.
It's still mostly used at Stanford for distributing keytabs, but it's
extensible and supports storing arbitrary secure data. We also use it
internally for storing database passwords, SSL private keys, and anything
else that needs to be kept secure and retrieved later. It supports a rich
ACL mechanism and namespace enforcement and is built on top of remctl for
security and confidentiality.
I did a bunch of development on it originally in 2006 and 2007, but then
it solved our immediate problems and I didn't have much time to work on
it. I used pre-1.0 versions because there was so much more on it that I
wanted to do. But we've been running it in production for years, and
while there's still tons more work that needs to be done, it really
deserves a 1.0 version. Jon Robertson converted the database backend to
DBIx::Class (which added PostreSQL support) and implemented schema
upgrading, which were two of the major things that were keeping me from
putting a 1.0 version on it. So it's time.
Other new features since the previous release include a change to the
default ACLs so that owners of objects can destroy them by default, a new
ldap-attr ACL type that checks whether the caller has a particular
attribute set in LDAP (which will let us use our entitlement system to
control access), and support for storing and managing key rotation for
WebAuth keyrings. There's also a new acl check command to
determine whether an ACL exists and a new comment field and command to
store an arbitrary comment about an object.
Our local naming policy has also gotten much more complex, so I turned it
into a module to provide an example to others of how to implement some
reasonably detailed namespace and object autocreation controls.
You can get the latest version from the wallet distribution page. I'm going to upgrade Debian packages to
experimental or unstable (depending on the freeze status) as soon as I get
a chance to write a bit more documentation and hopefully figure out
debconf integration with the database setup support inside Debian for the
server piece.
2013-03-28: lbcd 3.4.1
lbcd is an agent that we run on systems that use software load balancing.
It returns a weight and an increment that's used by the
lbnamed DNS server.
This is a minor feature release. The main point of this release is to add
support for an /etc/nolbcd file (or /usr/local/etc/nolbcd in the default
build) that's akin to /etc/nologin except it just drops the system from an
lbnamed load balance pool rather than denying all logins. Sometimes it's
more convenient to do that than to stop the lbcd daemon, particularly when
the latter is monitored by configuration management software.
Also in this release, the lbcd -t mode, which prints all the data
to standard output, is now documented and has been improved, and
lbcdclient no longer rewrites maximum weights to -1.
You can get the latest release from the lbcd
distribution page.
All posts