< April 2018 | Russ Allbery > Eagle's Path | June 2018 > |
As promised in the security advisory accompanying remctl 3.14, this release of remctl (and upcoming releases of C TAP Harness and rra-c-util) implements proper valgrind testing. In the process, I found and fixed a potential data loss bug: a server-side command accepting input on standard input that exited before consuming all of its input could have its output truncated due to a logic bug in the remctld server.
This release also adds some more protocol verification (I didn't see any bugs; this just tightens things out of paranoia) and much-improved maintainer support for static analysis and compiler warning checks.
You can get the latest release from the remctl distribution page.
DocKnot is the program that generates the top-level web pages and README files for my software packages (and eventually should take over generation of the LICENSE files, doing release management, and may eventually incorporate the tools for the macro language I use to generate my web pages).
For this release, I did a bit more research about the little badges that people have started displaying in their README files on GitHub and in a few other places (Rust's crates.io, for instance). DocKnot can now generate the Travis-CI badges, and also uses Shields.io for CPAN badges for things I upload to CPAN. I will probably add more as I work on other packages that have meaningful badges.
This release also moves the documentation of environment variables to set to enable more parts of the test suite to the testing section of README files. I have no idea why I put that in the requirements section originally; this should make more sense.
There are a few additions to support generating web pages for remctl, which previously wasn't using DocKnot, and a bug fix for README generation when there are additional bootstrapping instructions. (Previous versions miswrapped the resulting paragraph.)
DocKnot now requires Perl 5.24 or later, since right now I'm the only user
and don't need to support anything older than Debian stable, and that way
I can start using the neat new $ref->@*
Perl syntax instead of the
much noisier @{$ref}
syntax.
You can get the latest release from the DocKnot distribution page.
This is my test harness and C library for supporting TAP (Test Anything Protocol, the same used by Perl) tests for C projects.
The big change in this release is test options: a test list passed to
runtests
can now include space-separated options after the name of
the test to run. The two supported options are valgrind
, which
controls whether the test can be run under valgrind, and libtool
,
which further controls how the test is run under valgrind.
The background on this is a bit interesting.
I've had Makefile
targets to run test suites under valgrind for
various packages for a while, using valgrind's --trace-children=yes
option and running runtests
under valgrind. However, this has a
serious problem: valgrind tries to trace into every child, including shell
scripts or Perl scripts used in the test suite. This results in valgrind
logs that are basically always unuseful: lots of false positives or
uninteresting small memory leaks in standard system tools.
For projects that use Libtool, the situation becomes far worse, since
Libtool wraps all the executables in the build tree with shell scripts
that set up an appropriate library path. valgrind traces into those shell
scripts and all the various programs they run. As a result, running
valgrind with --trace-children=yes
on a non-trivial test suite such
as remctl's could easily result in 10,000
or more valgrind log files, most of which were just noise.
As a result, while the valgrind target was there, I rarely ran it or tried to dig through the results. And that, in turn, led to security vulnerabilities like the recent one in remctl.
Test options are a more comprehensive fix. When the valgrind
test
option is set, C TAP Harness will look at the C_TAP_VALGRIND
environment variable. If it is set, that test will be run, directly by
the harness, using the command given in that environment variable. A
typical setting might be:
VALGRIND_COMMAND = $(PATH_VALGRIND) --leak-check=full \ --log-file=$(abs_top_builddir)/tests/tmp/valgrind/log.%p \ --suppressions=$(abs_top_srcdir)/tests/data/valgrind.supp
and then a Makefile
target like:
# Used by maintainers to run the main test suite under valgrind. check-valgrind: $(check_PROGRAMS) rm -rf $(abs_top_builddir)/tests/tmp mkdir $(abs_top_builddir)/tests/tmp mkdir $(abs_top_builddir)/tests/tmp/valgrind C_TAP_VALGRIND="$(VALGRIND_COMMAND)" \ C_TAP_LIBTOOL="$(top_builddir)/libtool" \ tests/runtests -l '$(abs_top_srcdir)/tests/TESTS'
(this is take from the current version of remctl). Note that
--trace-children=yes
can be omitted, which avoids recursing into
any small helper programs that tests run, and only tests with the
valgrind
option set will be run under valgrind (the rest will be
run as normal). This cuts down massively on the noise. Tests that
explicitly want to invoke valgrind on some other program can use the
presence of C_TAP_VALGRIND
in the environment to decide whether to
do that and what command to use.
The C_TAP_LIBTOOL
setting above is the other half of the fix.
Packages that use Libtool may have tests that are Libtool-generated shell
wrappers, so just using the above would still run valgrind on a shell
script. But if the libtool
test option is also set, C TAP Harness
now knows to invoke the test with libtool --mode=execute
and then
the valgrind command, which causes Libtool to expand all of the shell
wrappers to actual binaries first and then run valgrind only on the actual
binary. The path to libtool is taken from the C_TAP_LIBTOOL
environment variable.
The final piece is an additional test that scans the generated valgrind log files. That piece is part of rra-c-util, so I'll talk about it with that release.
There are two more changes in this release. First, borrowing an idea from
Rust, diagnostics from test failures are now reported as the left value
and the right value, instead of the seen and wanted values. This allows
one to write tests without worrying about the order of arguments to the
is_*
functions, which turned out to be hard to remember and mostly
a waste of energy to keep consistent. And second, I fixed an old bug with
NULL handling in is_string
that meant a NULL might compare as equal
to a literal string "(null)"
. This test is now done properly.
The relationship between C TAP Harness and rra-c-util got a bit fuzzier in this release since I wanted to import more standard tests from rra-c-util and ended up importing the Perl test framework. This increases the amount of stuff that's in the C TAP Harness distribution that's canonically maintained in rra-c-util. I considered moving the point of canonical maintenance to C TAP Harness, but it felt rather out of scope for this package, so decided to live with it. Maybe I should just merge the two packages, but I know a lot of people who use pieces of C TAP Harness but don't care about rra-c-util, so it seems worthwhile to keep them separate.
I did replace the tests/docs/pod-t
and
tests/docs/pod-spelling-t
tests in C TAP Harness with the
rra-c-util versions in this release since I imported the Perl test
framework anyway. I think most users of the old C TAP Harness versions
will be able to use the rra-c-util versions without too much difficulty,
although it does require importing the Perl modules from rra-c-util into a
tests/tap/perl
directory.
You can get the latest release from the C TAP Harness distribution page.
This is my collection of portability libraries and test suite support for my C projects.
The big news in this release was already discussed in the corresponding
C TAP Harness release: much better support for valgrind
checking. Not only has valgrind been enabled in this package to check the
portability and utility layers, but the TAP helper library has been
updated to support spawning remctld with valgrind for remctl tests. (The
primary consumer of this support will be the remctl package, of course.) This release also includes a
valgrind/logs
check that is meant to run last in a test list and
scans a directory of valgrind logs, reporting test failures for any that
contain valgrind errors. If the test suite isn't run under valgrind, this
test is just skipped.
Some other fixes in this release:
Fix misplaced va_end
in putil_log_failure
.
Avoid testing the client IP address in network_bind_all
tests.
Some hosts in the compiler farm for open source testing have a weird
network configuration where localhost packets appear to come from an
IP address other than 127.0.0.1, resulting in spurious failures.
Define UINT32_MAX on systems that don't have it.
Skip more Autoconf and Automake files in the all_files
function
of Test::RRA::Automake.
This release also contains a few other, more minor fixes, and switches the license of all of my public domain source files to the FSF all-permissive license in the name of standardizing on licenses registered with the SPDX project and vetted by actual lawyers.
You can get the latest release from the rra-c-util distribution page.
podlators is the CPAN distribution that contains Pod::Man, Pod::Text, and its subclasses, used for rendering Perl documentation as man pages or text files. It's been part of Perl core for a very long time.
This release cleans up a bunch of long-standing bugs in Pod::Text and its
subclasses thanks to a series of excellent bug reports from eponymous
alias. The default value of the sentence option was documented
incorrectly, the width option was ignored in Pod::Text::Termcap, that
module also worked incorrectly when COLUMNS
wasn't set in the
environment, and both Pod::Text::Termcap and Pod::Text::Color had various
problems with wrapping that are now fixed. Long =item text was miswrapped
due to an incorrect length calculation, and text attributes are now
cleared at the end of each line and then reapplied to work better with
common pagers.
In addition, the none
value for the errors
option of
Pod::Man and Pod::Text now works correctly, thanks to a bug report from
Olly Betts.
You can get the latest version from the podlators distribution page.
This is rather late since I got distracted by various other things including, ironically, releasing a bunch of software. This is for April, so doesn't include the releases from this month.
The main release I worked on was remctl
3.14, which fixed a security bug introduced in 3.12 with the sudo
configuration option. This has since been replaced by 3.15, which has
more thorough maintainer testing infrastructure to hopefully prevent this
from happening again.
I also did the final steps of the release process for INN 2.6.2, although as usual Julien ÉLIE did all of the hard work.
On the Debian side, I uploaded a new rssh package for the migration to GitLab (salsa.debian.org). I have more work to do on that front, but haven't yet had the time. I've been prioritizing some of my own packages over doing more general Debian work.
Finally, I looked at my Perl modules on CPANTS (the CPAN testing service) and made note of a few things I need to fix, plus filed a couple of bugs for display issues (one of which turned out to be my fault and fixed in Git). I also did a bit of research on the badges that people in the Rust community use in their documentation and started adding support to DocKnot, some of which made it into the subsequent release I did this month.
Hm, I haven't done one of these in a while. Well, time to alienate future employers and make awkward mistakes in public that I have to explain if I ever run for office! (Spoiler: I'm highly unlikely to ever run for office.)
This is only of direct interest to California residents. To everyone else, RIP your feed reader, and I'm sorry for the length. (My hand-rolled blog software doesn't do cut tags.) I'll spare you all the drill-down into the Bay Area regional offices. (Apparently we elect our coroner, which makes no sense to me.)
I'm not explaining these because this is already much too long; those who aren't in California and want to follow along can see the voter guide.
Proposition 68: YES. Still a good time to borrow money, and what we're borrowing money for here seems pretty reasonable. State finances are in reasonable shape; we have the largest debt of any state for the obvious reason that we have the most people and the most money.
Proposition 69: YES. My instinct is to vote no because I have a general objection to putting restrictions on how the state manages its budget. I don't like dividing tax money into locked pools for the same reason that I stopped partitioning hard drives. That said, this includes public transit in the spending pool from gasoline taxes (good), the opposition is incoherent, and there are wide-ranging endorsements. That pushed me to yes on the grounds that maybe all these people understand something about budget allocations that I don't.
Proposition 70: NO. This is some sort of compromise with Republicans because they don't like what cap-and-trade money is being spent on (like high-speed rail) and want a say. If I wanted them to have a say, I'd vote for them. There's a reason why they have to resort to backroom tricks to try to get leverage over laws in this state, and it's not because they have good ideas.
Proposition 71: YES. Entirely reasonable change to say that propositions only go into effect after the election results are final. (There was a real proposition where this almost caused a ton of confusion, and prompted this amendment.)
Proposition 72: YES. I'm grumbling about this because I think we should get rid of all this special-case bullshit in property taxes and just readjust them regularly. Unfortunately, in our current property tax regime, you have to add more exemptions like this because otherwise the property tax hit (that would otherwise not be incurred) is so large that it kills the market for these improvements. Rainwater capture is to the public benefit in multiple ways, so I'll hold my nose and vote for another special exception.
US Senator: Kevin de León. I'll vote for Feinstein in the general against any Republican, and she's way up on de León in the polls, but there's no risk in voting for the more progressive candidate here since there's no chance Feinstein won't get the most votes in the primary. De León is a more solidly progressive candidate than Feinstein. I'd love to see a general election between the two of them.
I'm omitting all the unopposed ones, and all the ones where there's only one Democrat running in the primary. (I'm not going to vote for any Republican except for one exception noted below, and third parties in the US are unbelievably dysfunctional and not ready to govern.) For those outside the state, California has a jungle primary where the top two vote-getters regardless of party go to the general election, so this is more partisan and more important than other state primaries.
Governor: Delaine Eastin. One always has to ask, in our bullshit voting system, whether one has to vote tactically instead of for the best candidate. But, looking at polling, I think there's no chance Gavin Newsom (the second-best candidate and the front-runner) won't advance to the general election, so I get to vote for the candidate I actually want to win, even though she's probably not going to. Eastin is by far the most progressive candidate running who actually has the experience required to be governor. (Spoiler: Newsom is going to win, and I'll definitely vote for him in the general against Villaraigosa.)
Lieutenant Governor: Eleni Kounalakis. She and Bleich are the strongest candidates. I don't see a ton of separation between them, but Kounalakis's endorsements are a bit stronger for me. She's also the one candidate who has a specific statement about what she plans to do with the lieutenant governor role of oversight over the university system, which is almost its only actual power. (This political office is stupid and we should abolish it.)
Secretary of State: Alex Padilla. I agree more with Ruben Major's platform (100% paper ballots is the correct security position), but he's an oddball outsider and I don't think he can accomplish as much. Padilla has an excellent track record as the incumbant and is doing basically the right things, just less dramatically.
Treasurer: Fiona Ma. I like Vivek Viswanathan and support his platform, but Fiona Ma has a lot more political expertise and I think will be more effective. I look forward to voting for Viswanathan for something else someday.
Attorney General: Dave Jones. Xavier Becerra hasn't been doing a bad job fighting off bad federal policy, but that seems to be all that he's interested in, and he's playing partisan games with the office. He has an air of amateurishness and political hackery. Dave Jones holds the same positions in a more effective way, is more professional, and has done a good job as Insurance Commissioner.
Insurance Commissioner: Steve Poizner. I'm going to vote for the (former) Republican here. Poizner expressed some really bullshit views on immigration when he ran for governor (which he's apologized for). I wouldn't support him for a more political office. But he was an excellent insurance commissioner (see, for instance, the response to Blue Cross's rate increase request). I'm closer to Ricardo Lara politically, but in his statements to the press he comes across as a clown: self-driving car insurance problems, cannabis insurance, climate change insurance, and a bunch of other nonsense that makes me think he doesn't understand the job. The other democrat, Mahmood, seems like less of a partisan hack, but he's a virtual unknown. If this were an important partisan office, I'd hold my nose and vote for Lara, but the job of insurance commissioner is more to be an auditor and negotiator, and Poizner was really good at it.
Superintendent of Public Instruction: Tony Thurmond. The other front-runner is Marshall Tuck, who is a charter school advocate. I hate charter schools with the passion of a burning sun.
Regional Measure 3: YES. Even more hyper-local than the rest of this post, but mentioning it because it was a narrow call. Bridge tolls are regressive, and I'm not a big fan of raising them as opposed to, say, increasing property taxes (yes please) or income taxes. That said, taxing cars to pay for (largely) public transit is the direction the money should flow. It was thinly balanced for me, but the thrust of the projects won out over the distaste at the regressive tax.
Okay, yes, I've been buying a lot of books. I seem to be ensuring the rate of acquisition outpaces the rate of consumption even though I've gotten back into reading daily on the train. Well, also, Diane Duane had a crisis and therefore a book sale, so I picked up a whole pile of stuff that I don't have an immediate intention to read.
I still have three written but not yet posted reviews to go through, which are waiting on me finding time to do some editing, and another finished book to review. I'm finishing more non-fiction lately than fiction, possibly due to a less-than-ideal choice of the fiction book for my mood.
The problem with non-fiction is that non-fiction authors keep recommending other interesting-sounding books to read!
Ilona Andrews — On the Edge (sff)
Catherine Asaro (ed.) — Irresistible Forces (sff anthology)
Isaac Asimov (ed.) — The New Hugo Winners (sff anthology)
Fredrik Backman — Beartown (mainstream)
Steven Brust — Good Guys (sff)
Steven Brust & Skyler White — The Skill of Our Hands (sff)
Jo Clayton — Skeen's Leap (sff)
Jo Clayton — Skeen's Return (sff)
Jo Clayton — Skeen's Search (sff)
Diane Duane — So You Want to Be a Wizard (sff)
Diane Duane — Deep Wizardry (sff)
Diane Duane — High Wizardry (sff)
Diane Duane — A Wizard Abroad (sff)
Diane Duane — The Wizard's Dilemma (sff)
Diane Duane — A Wizard Alone (sff)
Diane Duane — Wizard's Holiday (sff)
Diane Duane — Wizards at War (sff)
Diane Duane — A Wizard of Mars (sff)
Diane Duane — Tale of the Five (sff)
Diane Duane — The Big Meow (sff)
Charles Duhigg — The Power of Habit (nonfiction)
Max Gladstone — Four Roads Cross (sff)
Max Gladstone — The Ruin of Angels (sff)
Alison Green — Ask a Manager (nonfiction)
Nicola Griffith — So Lucky (mainstream)
Dorothy J. Heydt — The Witch of Syracuse (sff)
N.K. Jemisin — The Awakened Kingdom (sff)
Richard Kadrey — From Myst to Riven (nonfiction)
T. Kingfisher — The Wonder Engine (sff)
Ilana C. Myer — Last Song Before Night (sff)
Cal Newport — Deep Work (nonfiction)
Cal Newport — So Good They Can't Ignore You (nonfiction)
Emilie Richards — When We Were Sisters (mainstream)
Graydon Saunders — The Human Dress (sff)
Bruce Schneier — Data and Goliath (nonfiction)
Brigid Schulte — Overwhelmed (nonfiction)
Rivers Solomon — An Unkindness of Ghosts (sff)
Douglas Stone & Sheila Heen — Thanks for the Feedback (nonfiction)
Jodi Taylor — Just One Damned Thing After Another (sff)
Catherynne M. Valente — Space Opera (sff)
Phew.
You'll notice a few in here that I've already read and reviewed.
The anthologies, the Backman, and a few others are physical books my parents were getting rid of.
So much good stuff in there I really want to read! And of course I've now started various other personal projects that don't involve spending all of my evenings and weekends reading.
< April 2018 | Russ Allbery > Eagle's Path | June 2018 > |