Posts for May 2018

2018-05-05: remctl 3.15

As promised in the security advisory accompanying remctl 3.14, this release of remctl (and upcoming releases of C TAP Harness and rra-c-util) implements proper valgrind testing. In the process, I found and fixed a potential data loss bug: a server-side command accepting input on standard input that exited before consuming all of its input could have its output truncated due to a logic bug in the remctld server.

This release also adds some more protocol verification (I didn't see any bugs; this just tightens things out of paranoia) and much-improved maintainer support for static analysis and compiler warning checks.

You can get the latest release from the remctl distribution page.

2018-05-05: DocKnot 1.05

DocKnot is the program that generates the top-level web pages and README files for my software packages (and eventually should take over generation of the LICENSE files, doing release management, and may eventually incorporate the tools for the macro language I use to generate my web pages).

For this release, I did a bit more research about the little badges that people have started displaying in their README files on GitHub and in a few other places (Rust's crates.io, for instance). DocKnot can now generate the Travis-CI badges, and also uses Shields.io for CPAN badges for things I upload to CPAN. I will probably add more as I work on other packages that have meaningful badges.

This release also moves the documentation of environment variables to set to enable more parts of the test suite to the testing section of README files. I have no idea why I put that in the requirements section originally; this should make more sense.

There are a few additions to support generating web pages for remctl, which previously wasn't using DocKnot, and a bug fix for README generation when there are additional bootstrapping instructions. (Previous versions miswrapped the resulting paragraph.)

DocKnot now requires Perl 5.24 or later, since right now I'm the only user and don't need to support anything older than Debian stable, and that way I can start using the neat new $ref->@* Perl syntax instead of the much noisier @{$ref} syntax.

You can get the latest release from the DocKnot distribution page.

2018-05-06: C TAP Harness 4.3

This is my test harness and C library for supporting TAP (Test Anything Protocol, the same used by Perl) tests for C projects.

The big change in this release is test options: a test list passed to runtests can now include space-separated options after the name of the test to run. The two supported options are valgrind, which controls whether the test can be run under valgrind, and libtool, which further controls how the test is run under valgrind.

The background on this is a bit interesting.

I've had Makefile targets to run test suites under valgrind for various packages for a while, using valgrind's --trace-children=yes option and running runtests under valgrind. However, this has a serious problem: valgrind tries to trace into every child, including shell scripts or Perl scripts used in the test suite. This results in valgrind logs that are basically always unuseful: lots of false positives or uninteresting small memory leaks in standard system tools.

For projects that use Libtool, the situation becomes far worse, since Libtool wraps all the executables in the build tree with shell scripts that set up an appropriate library path. valgrind traces into those shell scripts and all the various programs they run. As a result, running valgrind with --trace-children=yes on a non-trivial test suite such as remctl's could easily result in 10,000 or more valgrind log files, most of which were just noise.

As a result, while the valgrind target was there, I rarely ran it or tried to dig through the results. And that, in turn, led to security vulnerabilities like the recent one in remctl.

Test options are a more comprehensive fix. When the valgrind test option is set, C TAP Harness will look at the C_TAP_VALGRIND environment variable. If it is set, that test will be run, directly by the harness, using the command given in that environment variable. A typical setting might be:

VALGRIND_COMMAND = $(PATH_VALGRIND) --leak-check=full             \
        --log-file=$(abs_top_builddir)/tests/tmp/valgrind/log.%p  \
        --suppressions=$(abs_top_srcdir)/tests/data/valgrind.supp

and then a Makefile target like:

# Used by maintainers to run the main test suite under valgrind.
check-valgrind: $(check_PROGRAMS)
        rm -rf $(abs_top_builddir)/tests/tmp
        mkdir $(abs_top_builddir)/tests/tmp
        mkdir $(abs_top_builddir)/tests/tmp/valgrind
        C_TAP_VALGRIND="$(VALGRIND_COMMAND)"                    \
            C_TAP_LIBTOOL="$(top_builddir)/libtool"             \
            tests/runtests -l '$(abs_top_srcdir)/tests/TESTS'

(this is take from the current version of remctl). Note that --trace-children=yes can be omitted, which avoids recursing into any small helper programs that tests run, and only tests with the valgrind option set will be run under valgrind (the rest will be run as normal). This cuts down massively on the noise. Tests that explicitly want to invoke valgrind on some other program can use the presence of C_TAP_VALGRIND in the environment to decide whether to do that and what command to use.

The C_TAP_LIBTOOL setting above is the other half of the fix. Packages that use Libtool may have tests that are Libtool-generated shell wrappers, so just using the above would still run valgrind on a shell script. But if the libtool test option is also set, C TAP Harness now knows to invoke the test with libtool --mode=execute and then the valgrind command, which causes Libtool to expand all of the shell wrappers to actual binaries first and then run valgrind only on the actual binary. The path to libtool is taken from the C_TAP_LIBTOOL environment variable.

The final piece is an additional test that scans the generated valgrind log files. That piece is part of rra-c-util, so I'll talk about it with that release.

There are two more changes in this release. First, borrowing an idea from Rust, diagnostics from test failures are now reported as the left value and the right value, instead of the seen and wanted values. This allows one to write tests without worrying about the order of arguments to the is_* functions, which turned out to be hard to remember and mostly a waste of energy to keep consistent. And second, I fixed an old bug with NULL handling in is_string that meant a NULL might compare as equal to a literal string "(null)". This test is now done properly.

The relationship between C TAP Harness and rra-c-util got a bit fuzzier in this release since I wanted to import more standard tests from rra-c-util and ended up importing the Perl test framework. This increases the amount of stuff that's in the C TAP Harness distribution that's canonically maintained in rra-c-util. I considered moving the point of canonical maintenance to C TAP Harness, but it felt rather out of scope for this package, so decided to live with it. Maybe I should just merge the two packages, but I know a lot of people who use pieces of C TAP Harness but don't care about rra-c-util, so it seems worthwhile to keep them separate.

I did replace the tests/docs/pod-t and tests/docs/pod-spelling-t tests in C TAP Harness with the rra-c-util versions in this release since I imported the Perl test framework anyway. I think most users of the old C TAP Harness versions will be able to use the rra-c-util versions without too much difficulty, although it does require importing the Perl modules from rra-c-util into a tests/tap/perl directory.

You can get the latest release from the C TAP Harness distribution page.

2018-05-06: rra-c-util 7.1

This is my collection of portability libraries and test suite support for my C projects.

The big news in this release was already discussed in the corresponding C TAP Harness release: much better support for valgrind checking. Not only has valgrind been enabled in this package to check the portability and utility layers, but the TAP helper library has been updated to support spawning remctld with valgrind for remctl tests. (The primary consumer of this support will be the remctl package, of course.) This release also includes a valgrind/logs check that is meant to run last in a test list and scans a directory of valgrind logs, reporting test failures for any that contain valgrind errors. If the test suite isn't run under valgrind, this test is just skipped.

Some other fixes in this release:

This release also contains a few other, more minor fixes, and switches the license of all of my public domain source files to the FSF all-permissive license in the name of standardizing on licenses registered with the SPDX project and vetted by actual lawyers.

You can get the latest release from the rra-c-util distribution page.

2018-05-07: podlators 4.11

podlators is the CPAN distribution that contains Pod::Man, Pod::Text, and its subclasses, used for rendering Perl documentation as man pages or text files. It's been part of Perl core for a very long time.

This release cleans up a bunch of long-standing bugs in Pod::Text and its subclasses thanks to a series of excellent bug reports from eponymous alias. The default value of the sentence option was documented incorrectly, the width option was ignored in Pod::Text::Termcap, that module also worked incorrectly when COLUMNS wasn't set in the environment, and both Pod::Text::Termcap and Pod::Text::Color had various problems with wrapping that are now fixed. Long =item text was miswrapped due to an incorrect length calculation, and text attributes are now cleared at the end of each line and then reapplied to work better with common pagers.

In addition, the none value for the errors option of Pod::Man and Pod::Text now works correctly, thanks to a bug report from Olly Betts.

You can get the latest version from the podlators distribution page.

2018-05-19: Free software log (April 2018)

This is rather late since I got distracted by various other things including, ironically, releasing a bunch of software. This is for April, so doesn't include the releases from this month.

The main release I worked on was remctl 3.14, which fixed a security bug introduced in 3.12 with the sudo configuration option. This has since been replaced by 3.15, which has more thorough maintainer testing infrastructure to hopefully prevent this from happening again.

I also did the final steps of the release process for INN 2.6.2, although as usual Julien ÉLIE did all of the hard work.

On the Debian side, I uploaded a new rssh package for the migration to GitLab (salsa.debian.org). I have more work to do on that front, but haven't yet had the time. I've been prioritizing some of my own packages over doing more general Debian work.

Finally, I looked at my Perl modules on CPANTS (the CPAN testing service) and made note of a few things I need to fix, plus filed a couple of bugs for display issues (one of which turned out to be my fault and fixed in Git). I also did a bit of research on the badges that people in the Rust community use in their documentation and started adding support to DocKnot, some of which made it into the subsequent release I did this month.

2018-05-19: California state primary

Hm, I haven't done one of these in a while. Well, time to alienate future employers and make awkward mistakes in public that I have to explain if I ever run for office! (Spoiler: I'm highly unlikely to ever run for office.)

This is only of direct interest to California residents. To everyone else, RIP your feed reader, and I'm sorry for the length. (My hand-rolled blog software doesn't do cut tags.) I'll spare you all the drill-down into the Bay Area regional offices. (Apparently we elect our coroner, which makes no sense to me.)

Propositions

I'm not explaining these because this is already much too long; those who aren't in California and want to follow along can see the voter guide.

Proposition 68: YES. Still a good time to borrow money, and what we're borrowing money for here seems pretty reasonable. State finances are in reasonable shape; we have the largest debt of any state for the obvious reason that we have the most people and the most money.

Proposition 69: YES. My instinct is to vote no because I have a general objection to putting restrictions on how the state manages its budget. I don't like dividing tax money into locked pools for the same reason that I stopped partitioning hard drives. That said, this includes public transit in the spending pool from gasoline taxes (good), the opposition is incoherent, and there are wide-ranging endorsements. That pushed me to yes on the grounds that maybe all these people understand something about budget allocations that I don't.

Proposition 70: NO. This is some sort of compromise with Republicans because they don't like what cap-and-trade money is being spent on (like high-speed rail) and want a say. If I wanted them to have a say, I'd vote for them. There's a reason why they have to resort to backroom tricks to try to get leverage over laws in this state, and it's not because they have good ideas.

Proposition 71: YES. Entirely reasonable change to say that propositions only go into effect after the election results are final. (There was a real proposition where this almost caused a ton of confusion, and prompted this amendment.)

Proposition 72: YES. I'm grumbling about this because I think we should get rid of all this special-case bullshit in property taxes and just readjust them regularly. Unfortunately, in our current property tax regime, you have to add more exemptions like this because otherwise the property tax hit (that would otherwise not be incurred) is so large that it kills the market for these improvements. Rainwater capture is to the public benefit in multiple ways, so I'll hold my nose and vote for another special exception.

Federal Offices

US Senator: Kevin de León. I'll vote for Feinstein in the general against any Republican, and she's way up on de León in the polls, but there's no risk in voting for the more progressive candidate here since there's no chance Feinstein won't get the most votes in the primary. De León is a more solidly progressive candidate than Feinstein. I'd love to see a general election between the two of them.

State Offices

I'm omitting all the unopposed ones, and all the ones where there's only one Democrat running in the primary. (I'm not going to vote for any Republican except for one exception noted below, and third parties in the US are unbelievably dysfunctional and not ready to govern.) For those outside the state, California has a jungle primary where the top two vote-getters regardless of party go to the general election, so this is more partisan and more important than other state primaries.

Governor: Delaine Eastin. One always has to ask, in our bullshit voting system, whether one has to vote tactically instead of for the best candidate. But, looking at polling, I think there's no chance Gavin Newsom (the second-best candidate and the front-runner) won't advance to the general election, so I get to vote for the candidate I actually want to win, even though she's probably not going to. Eastin is by far the most progressive candidate running who actually has the experience required to be governor. (Spoiler: Newsom is going to win, and I'll definitely vote for him in the general against Villaraigosa.)

Lieutenant Governor: Eleni Kounalakis. She and Bleich are the strongest candidates. I don't see a ton of separation between them, but Kounalakis's endorsements are a bit stronger for me. She's also the one candidate who has a specific statement about what she plans to do with the lieutenant governor role of oversight over the university system, which is almost its only actual power. (This political office is stupid and we should abolish it.)

Secretary of State: Alex Padilla. I agree more with Ruben Major's platform (100% paper ballots is the correct security position), but he's an oddball outsider and I don't think he can accomplish as much. Padilla has an excellent track record as the incumbant and is doing basically the right things, just less dramatically.

Treasurer: Fiona Ma. I like Vivek Viswanathan and support his platform, but Fiona Ma has a lot more political expertise and I think will be more effective. I look forward to voting for Viswanathan for something else someday.

Attorney General: Dave Jones. Xavier Becerra hasn't been doing a bad job fighting off bad federal policy, but that seems to be all that he's interested in, and he's playing partisan games with the office. He has an air of amateurishness and political hackery. Dave Jones holds the same positions in a more effective way, is more professional, and has done a good job as Insurance Commissioner.

Insurance Commissioner: Steve Poizner. I'm going to vote for the (former) Republican here. Poizner expressed some really bullshit views on immigration when he ran for governor (which he's apologized for). I wouldn't support him for a more political office. But he was an excellent insurance commissioner (see, for instance, the response to Blue Cross's rate increase request). I'm closer to Ricardo Lara politically, but in his statements to the press he comes across as a clown: self-driving car insurance problems, cannabis insurance, climate change insurance, and a bunch of other nonsense that makes me think he doesn't understand the job. The other democrat, Mahmood, seems like less of a partisan hack, but he's a virtual unknown. If this were an important partisan office, I'd hold my nose and vote for Lara, but the job of insurance commissioner is more to be an auditor and negotiator, and Poizner was really good at it.

Superintendent of Public Instruction: Tony Thurmond. The other front-runner is Marshall Tuck, who is a charter school advocate. I hate charter schools with the passion of a burning sun.

Local Measures

Regional Measure 3: YES. Even more hyper-local than the rest of this post, but mentioning it because it was a narrow call. Bridge tolls are regressive, and I'm not a big fan of raising them as opposed to, say, increasing property taxes (yes please) or income taxes. That said, taxing cars to pay for (largely) public transit is the direction the money should flow. It was thinly balanced for me, but the thrust of the projects won out over the distaste at the regressive tax.

Last modified and spun 2018-05-20