Testing PAM modules

I maintain a couple of PAM modules, pam-afs-session and pam-krb5. I'm also a big fan of test-driven development and automated regression testing, and have been slowly converting all of my software over to using a test suite. But therein lies a challenge, since PAM modules are hard to test.

First, Linux PAM doesn't offer a good way to run a module in a test environment. Specifically, it doesn't offer a way to use an alternative PAM configuration other than the system PAM configuration, so that one can load only the module being tested (or it and some glue modules to set things up). One has to resort to dedicated test systems that do things like symlink part of their PAM configuration into a shared build tree.

Second, even if this problem were solved, Linux PAM doesn't (rightfully) allow a lot of introspection and digging around in its guts. This makes it harder to test whether data items, environment variables, and so forth are set properly and make sense. It also doesn't provide facilities for intercepting logging output that would otherwise go to syslog.

So, about a year and a half ago (and then with substantial improvements a year ago), I wrote a fake PAM library to test pam-afs-session. This library provides all of the PAM API that's called by a PAM module (at least the modules I'm testing), but has a much-simplified interface and allows test cases to look through its internals and be sure the module did things properly.

This was great as far as it goes. But writing the wrapper code to invoke a PAM module entry point and then check its return status, its logging, and so forth was complicated, messy, and difficult to understand. It also didn't handle testing prompting and other important aspects of PAM modules. So earlier this fall I wrote the beginnings of a PAM test driver that would move as much of the complexity as possible into shared code and into configuration files, and since then I've been expanding and improving it until it could handle the major testing required for pam-krb5 (which is much more complex than pam-afs-session). Yesterday, I released the results as part of rra-c-util.

Here's an example configuration file, this one for a pam-krb5 test that simulates prompting the user for a password and then obtaining credentials (but not creating a ticket cache):

    [options]
        auth    = no_ccache
        account = no_ccache
        session = no_ccache

    [run]
        authenticate  = PAM_SUCCESS
        acct_mgmt     = PAM_SUCCESS
        open_session  = PAM_SUCCESS
        close_session = PAM_SUCCESS

    [prompts]
        echo_off = Password: |%p

    [output]
        INFO user %u authenticated as %u

That's a complete and fairly complex test. The test framework takes care of running the listed entry points, passing them the given PAM configuration, and checking the return status. It provides a conversation function that checks the prompt and its type and replies with the text after |. As you can see, it supports some escapes that are replaced by data passed into the test suite driver, in this case the password to use for testing. And it checks the logging output, including the priority level, and similarly supports escapes there.

Running simple tests is very easy and is doable with a tiny C program. Here's a complete test program from pam-krb5, one that tests functionality that doesn't require a valid password and hence doesn't need to do much setup:

    #include <config.h>
    #include <portable/system.h>
    #include <tests/fakepam/script.h>

    int
    main(void)
    {
        struct script_config config;

        plan_lazy();
        memset(&config, 0, sizeof(config));
        config.user = "root";
        run_script_dir("data/scripts/basic", &config);
        return 0;
    }

That's all there is to it. That finds every configuration file like the above in the tests/data/scripts/basic directory in the package source and runs through each one, checking the results, and outputing the results as TAP output (the protocol used for the Perl test suite, and also my C TAP Harness package).

There are, of course, other things that it can do: more substitutions, setting the authtok in advance of calling the module, and a callback so that one can inspect PAM state before the PAM session is shut down. I've started writing some documentation, although so far it primarily covers the configuration file syntax and the headers are the best reference for the API.

The source code of the pam-krb5 package are the best reference so far. The library itself is part of rra-c-util and, like the rest of that package, is designed to be copied into the package that uses it and built alongside it so that it can be linked easily with the test programs.

It's worth noting that there isn't a way to use it to test installed PAM modules that are already bound to the system libpam library. The way one builds test programs currently is to link the test program with the individual objects making up the PAM module and then link with this library and the TAP library, since that ensures the right symbols are called. But it may be possible to do something tricky using LD_PRELOAD if anyone feels inspired.