Posts for December 2013

2013-12-09: rra-c-util 4.12

This release of my collection of shared C, Perl, and Autoconf code fixes a bug in all the Autoconf macros that use the lib-helper framework for optional use of libraries. The --with flag without a path would result in yes/include and yes/lib to be added to the compiler and linker paths. It also adds Autoconf probles for Cyrus SASL libraries, contributed by Julien √ČLIE based on the INN macros.

This release also adds support for KADM5_MISSING_KRB5_CONF_PARAMS to portable/kadmin.h and the test_tmpdir function to Test::RRA::Automake. The latter works the same as it does in the C and shell TAP libraries.

Finally, the shared valgrind suppression file adds a suppression for the memory allocated by dlopen to store error messages for dlerror on Linux, which is apparently never freed.

You can get the latest release from the rra-c-util distribution page.

2013-12-09: krb5-sync 3.0

krb5-sync is the software we run at Stanford to synchronize principal information from a central Heimdal realm to Active Directory, allowing users to use either a Linux-based Kerberos environment or Active Directory with the same account and password.

The original intent of this release was to add a new feature to allow a subsidiary instance of an account in the MIT or Heimdal realm to be synchronized with the instance-less account in Active Directory. This allows, for example, an rra/windows instance to be used to set and maintain the password for an rra principal in Active Directory.

In the process of implementing that, though, I ended up doing a significant overall of the code, since the plugin architecture was quite awkward and dated. The code now uses the MIT Kerberos data structures in a more natural and native way, since MIT Kerberos has now added direct support for plugins of this sort. Kerberos contexts and Kerberos error codes are used uniformly throughout the plugin, which provides consistent and more robust error handling and reporting. I also significantly enhanced the test suite, although it still needs more work to test the core functionality that has complex external dependencies. This release also drops support for all versions of MIT Kerberos prior to 1.9, which required an external patch; to run krb5-sync 3.0, you should upgrade to a recent version of MIT Kerberos. This allowed me to drop support for the legacy API.

There are a couple of major backward-incompatible changes in this release (and both unfortunately are not handled automatically by the Debian package upgrade, since it's hard to find and safely modify KDC configuration). First, the ad_ldap_base configuration option is now mandatory when synchronizing account status and its meaning has changed. Previously, dc elements for the realm were appended to a provided partial base. Now, the complete DN of the root of the Active Directory tree should be provided. This is more flexible and more useful with a wider variety of Active Directory setups.

Second, I took advantage of the backward-incompatibilities to change the module name to from, since the latter sounded weirdly redundant and verbose when installed in the Kerberos plugin directory. This will require a configuration change to the plugin configuration for the KDC or kadmin server.

Also in this release are a couple of new options: ad_queue_only, which forces all changes to be queued for later processing instead of processed in real time, and syslog, which can be used to turn off the internal syslog logging of non-errors from the module. (This is mostly useful for test suites.)

Now, password changes are queued on any Active Directory failure, not just a few oddly-distinguished ones. The previous behavior was rather specific to Stanford's needs, and queuing all password changes shouldn't pose any problems.

Finally, the krb5-sync-backend utility program for manipulating the queued changes has been completely rewritten and is much cleaner. It now uses the Net::Remctl::Backend Perl module for command and option handling, so that module (provided with remctl 3.4 or later) must be installed. It also requires IPC::Run, which is available from CPAN. It uniformly supports a -d option to specify the queue location, and skips event files during processing that no longer exist.

You can get the latest release from krb5-sync distribution page.

2013-12-16: krb5-strength 2.2

krb5-strength provides a plugin and external password checking program implementation of password strength checking for Kerberos KDCs.

Management at Stanford has decided that we want to impose different character class restrictions by length, with longer passwords having fewer required character classes. This release therefore adds more comprehensive character class requirement support, including the ability to set varying requirements based on the length of the password.

This release also improves the cdbmake-wordlist utility, adding support for removing words longer than a maximum length and filtering out words that match a user-provided regular expression. It can also be run in filter mode to generate a new wordlist instead of a CDB file.

Finally, a file descriptor and memory leak in the embedded version of CrackLib has been fixed. (This was already fixed in the regular CrackLib release.)

You can get the latest version from the krb5-strength distribution page.

2013-12-17: lbcd 3.4.2

lbcd is the daemon run on individual systems that participate in DNS-based load-balanced pools using lbnamed. This is a portability release that (finally) switches the default API for user login information over to getutxent from getutent (required for Mac OS X) and enables building on FreeBSD and Debian GNU/kFreeBSD systems. Note that lbcd will only work on FreeBSD systems if the Linux-compatible /proc file system is mounted, but this appears to be a common configuration.

You can get the latest release from the lbcd distribution page.

2013-12-25: C TAP Harness 2.4

I always enjoy this time of year: lots of peace and quiet and time to work on whatever I feel like focusing on. That's been watching a lot of speedrunning and League of Legends, but also experimenting with systemd and upstart. I've now ported lbcd to both "properly," meaning that I make full use of their features as far as I currently understand them.

I have some more packaging work to do, and need to make a release, but of course I started fixing various other things since I was in the code anyway, and now I don't want to release it without some testing. And that prompted another digression, since I didn't have a good test framework for spawning the server and pounding on it.

That finally brings this journal entry to its actual topic: a new release of C TAP Harness. This release adds a new pair of functions, diag_file_add() and diag_file_remove(), which tell the TAP library to take the contents of log files as an additional source of diag() messages. This produces much nicer, and more readable, output from test cases that involve forking a background server that produces output to standard output and standard error. That output can be directed to a file and then included in the test output stream, properly tagged and in sequence with the result messages.

You can get the latest release from the C TAP Harness distribution page.

2013-12-27: rra-c-util 5.0

This package started as my collection of shared C utility code and has turned into the collection of all sorts of glue that I use across multiple packages.

The primary changes in this release are to the network utility layer. In order to add proper IPv6 support, I adopted the network_bind_* functions to take the socket type, which changed all the APIs (hence the major version bump). While I was at it, I also cleaned up some of the error reporting in the various functions to make them easier to use. There's a new network_wait_any function that does the select part of network_accept_any, making it useful for UDP servers.

This release also contains a new, generalized TAP add-on for spawning background processes used by test cases. Their standard output and standard error is now captured and merged into the test output stream using the new C TAP Harness support for files containing diagnostic messages. The remctl TAP add-on has been rewritten to use this framework.

Julien √ČLIE also adapted the Autoconf probes for Berkeley DB and zlib used in INN to the framework used for most probes in rra-c-util, and those probes are included in this release.

Finally, the vector utility library's free functions now accept (and do nothing with) NULL, which makes it easier to write cleanup functions.

You can get the latest release from the rra-c-util distribution page.

2013-12-28: lbcd 3.5.0

This turned out to be a more substantial release than I was originally planning.

lbcd is a small daemon that listens to UDP queries and returns system load and usage information and, optionally, service availability. It's designed to work with a software DNS-based load-balancing system that we use at Stanford (lbnamed), but it can also be used in other situations where you want to easily and quickly query the load of systems remotely.

Debian is currently debating which new init system to adopt, and lbcd was my test package for adding upstart and systemd support. The goal was to add proper, full support as described by upstream, making use of all relevant features, so that I could get some experience with a complete integration. That is complete in this version: lbcd now supports both systemd's and upstart's readiness synchronization mechanisms (using a -Z flag to raise SIGSTOP for upstart) and supports socket activation with systemd. (upstart's socket activation support is missing numerous features, including IPv6 support and, fatally, SOCK_DGRAM support.) It also installs systemd unit files directly when built on a system with systemd support, which should make life easier for use on Red Hat versions later regardless of what happens with Debian.

Of course, while doing that, I kept running across other things that needed to be fixed. For example, systemd's socket activation provides proper support for IPv6, so lbcd should as well. That's now present; that was something I'd been planning on doing for some time. It also supports a -f option to run in the foreground but still log to syslog, something needed by both upstart and systemd to avoid having to use PID files.

Since PID files are no longer necessary, lbcd no longer writes one by default (an idiosyncratic choice made by the previous maintainer), and also drops the -s and -r options to stop and restart itself. Adding these to each daemon was an interesting approach, but I think it's better to leave this to the init system.

While working on the code, I discovered that lbcd allowed the client to request any of the built-in service probes be run, which meant that a client could cause TCP connects to random local services. While this probably couldn't do any harm other than a DoS attack, it still seemed like a bad idea, and was a "feature" I didn't realize was there. Now, only services specified with the -w or -a options may be queried by a client.

I also finally implemented the -l option, which logs each client query, and improved lbcd's recognition of whether someone is on console to allow for modern display manager sessions.

The simple client included in the package, lbcdclient, has been completely rewritten using modern Perl. It supports long options, setting the timeout, setting the port, and returns an error on timeout. It no longer supports multiple servers to query on the command line, since the output just gets confusing and I don't think anyone used this feature. It also now supports IPv6 if IO::Socket::INET6 is available.

Finally, a typo that prevented compilation on Mac OS X has been fixed.

You can get the latest version from the lbcd distribution page.

2013-12-30: Init system evaluation thanks

Most people who follow Debian are probably aware that we're currently in the middle of a major debate over the future of Debian's init system. Over the past month, and particularly over the past week, I undertook an evaluation of two of the options in that debate, using one of my packages as a test bed. I also did a lot of reading, filed a few bugs, and asked a bunch of questions. That's an extension of the questions I've been asking and discussions I've been participating in around init systems for the past year plus.

I have been keeping my opinions and writeups confined to the relevant Techncial Committee bug report so that they can be read in context with their rebuttals. There are a lot of thoughtful and well-informed opinions here, of which mine are only one, and several of my conclusions have been inaccurate or at least not complete, so the context is important. You can find the full traffic in the Technical Committee bug. Most of the discussion, in a threaded view, is in the Technical Committee archives for December.

However, in one of the messages I sent today, I tried to show my appreciation for the people who have assisted with this evaluation over the past week, month, and years, and I think that section deserves broader distribution outside of the ongoing discussion. It is therefore included below.

Throughout this evaluation process, my interactions with upstart and systemd upstream developers and Debian packagers have been uniformly excellent. Bug reports filed against both systemd and upstart have received excellent and timely response, and all involved have been quite willing to explain things I've misunderstood, correct my false starts, and discuss technical and practical aspects of their designs.

I was particularly impressed by the clear effort that the systemd and upstart maintainers in Debian have put into fully integrating their init systems in such a way that makes them easy to test and use with existing Debian packages. This includes but is not limited to update-rc.d support, invoke-rc.d support, status synchronization with sysvinit, past Policy discussion, and attention to upgrade paths and init-switching use cases.

I also want to particularly thank the OpenRC upstream development team for their involvement in this process and their contributions to the discussion. I personally don't think that package is a good match for Debian's needs on Linux, but that's through no fault of the people involved, and I think they would be an excellent upstream if that package looked like a good fit for the needs of any of Debian's non-Linux ports.

I also want to thank Petter Reinholdtsen, Roger Leigh, and everyone else who has worked on the sysvinit package over the years, the insserv conversion to dependency-based boot, and the inclusion of LSB support. If it weren't for their hard work, we would be in a far worse position than we are today. It's often hard to see people discussing the inadequacies of something into which you put years of hard work. I want to call attention to their long-term contributions to the distribution, and to the number of Debian systems that have booted through their efforts over the years.

Last modified and spun 2017-02-20