2017-10-15: Free software log (September 2017)
I said that I was going to start writing these regularly, so I'm going to
stick to it, even when the results are rather underwhelming. One of the
goals is to make the time for more free software work, and I do better at
doing things that I record.
The only piece of free software work for September was that I made
rra-c-util compile cleanly with the
Clang static analyzer. This was fairly tedious work that mostly involved
unconfusing the compiler or converting (semi-intentional) crashes into
explicit asserts, but it unblocks using the Clang static analyzer as part
of the automated test suite of my other projects that are downstream of
rra-c-util.
One of the semantic changes I made was that the vector utilities in
rra-c-util (which maintain a resizable array of strings) now always
allocate room for at least one string pointer. This wastes a small amount
of memory for empty vectors that are never used, but ensures that the
strings struct member is always valid. This isn't, strictly
speaking, a correctness fix, since all the checks were correct, but after
some thought, I decided that humans might have the same problem that the
static analyzer had. It's a lot easier to reason about a field that's
never NULL. Similarly, the replacement function for a missing
reallocarray now does an allocation of size 1 if given a size of 0,
just to avoid edge case behavior. (I'm sure the behavior of a realloc
with size 0 is defined somewhere in the C standard, but if I have to look
it up, I'd rather not make a human reason about it.)
I started on, but didn't finish, making rra-c-util compile without Clang
warnings (at least for a chosen set of warnings). By far the hardest
problem here are the Clang warnings for comparisons between unsigned and
signed integers. In theory, I like this warning, since it's the cause of
a lot of very obscure bugs. In practice, gah does C ever do this all over
the place, and it's incredibly painful to avoid. (One of the biggest
offenders is write, which returns a ssize_t that you almost
always want to compare against a size_t.) I did a bunch of
mechanical work, but I now have a lot of bits of code like:
if (status < 0)
return;
written = (size_t) status;
if (written < avail)
buffer->left += written;
which is ugly and unsatisfying. And I also have a ton of casts, such as
with:
buffer_resize(buffer, (size_t) st.st_size + used);
since st.st_size is an off_t, which may be signed. This is
all deeply unsatisfying and ugly, and I think it makes the code moderately
harder to read, but I do think the warning will potentially catch bugs and
even security issues.
I'm still torn. Maybe I can find some nice macros or programming styles
to avoid the worst of this problem. It definitely requires more thought,
rather than just committing this huge mechanical change with lots of ugly
code.
Mostly, this kind of nonsense makes me want to stop working on C code and
go finish learning Rust....
Anyway, apart from work, the biggest thing I managed to do last month that
was vaguely related to free software was upgrading my personal servers to
stretch (finally). That mostly went okay; only a few things made it
unnecessarily exciting.
The first was that one of my systems had a very tiny / partition that was
too small to hold the downloaded debs for the upgrade, so I had to resize
it (VM disk, partition, and file system), and that was a bit exciting
because it has an old-style DOS partition table that isn't aligned (hmmm,
which is probably why disk I/O is so slow on those VMs), so I had to use
the obsolete fdisk -c=dos mode because I wasn't up for replacing
the partition right then.
The second was that my first try at an upgrade died with a segfault during
the libc6 postinst and then every executable segfaulted. A mild panic and
a rescue disk later (and thirty minutes and a lot of swearing), I tracked
the problem down to libc6-xen. Nothing in the dependency structure
between jessie and stretch forces libc6-xen to be upgraded in lockstep or
removed, but it's earlier in the search path. So ld.so gets upgraded, and
then finds the old libc6 from the libc6-xen package, and the mismatch
causes immediate segfaults. A chroot dpkg --purge from the rescue
disk solved the problem as soon as I knew what was going on, but that was
a stressful half-hour.
The third problem was something I should have known was going to be an
issue: an old Perl program that does some internal stuff for one of the
services I ran had a defined @array test that has been warning for
eons and that I never fixed. That became a full syntax error with the
most recent Perl, and then I fixed it incorrectly the first time and had a
bunch of trouble tracking down what I'd broken. All sorted out now, and
everything is happily running stretch. (ejabberd, which other folks had
mentioned was a problem, went completely smoothly, although I suspect I
now have too many of the plugin packages installed and should do a
purging.)
2017-10-16: Bundle haul
Confession time: I started making these posts (eons ago) because a close
friend did as well, and I enjoyed reading them. But the main reason why I
continue is because the primary way I have to keep track of the books I've
bought and avoid duplicates is, well, grep on these posts.
I should come up with a non-bullshit way of doing this, but time to do
more elegant things is in short supply, and, well, it's my blog. So I'm
boring all of you who read this in various places with my internal
bookkeeping. I do try to at least add a bit of commentary.
This one will be more tedious than most since it includes five separate
Humble Bundles, which
increases the volume a lot. (I just realized I'd forgotten to record
those purchases from the past several months.)
First, the individual books I bought directly:
Ilona Andrews — Sweep in Peace (sff)
Ilona Andrews — One Fell Sweep (sff)
Steven Brust — Vallista (sff)
Nicky Drayden — The Prey of Gods (sff)
Meg Elison — The Book of the Unnamed Midwife (sff)
Pat Green — Night Moves (nonfiction)
Ann Leckie — Provenance (sff)
Seanan McGuire — Once Broken Faith (sff)
Seanan McGuire — The Brightest Fell (sff)
K Arsenault Rivera — The Tiger's Daughter (sff)
Matthew Walker — Why We Sleep (nonfiction)
Some new books by favorite authors, a few new releases I heard good things
about, and two (Night Moves and Why We Sleep) from
references in on-line articles that impressed me.
The books from security bundles (this is mostly work reading, assuming
I'll get to any of it), including a blockchain bundle:
Wil Allsop — Unauthorised Access (nonfiction)
Ross Anderson — Security Engineering (nonfiction)
Chris Anley, et al. — The Shellcoder's Handbook (nonfiction)
Conrad Barsky & Chris Wilmer — Bitcoin for the Befuddled
(nonfiction)
Imran Bashir — Mastering Blockchain (nonfiction)
Richard Bejtlich — The Practice of Network Security
(nonfiction)
Kariappa Bheemaiah — The Blockchain Alternative (nonfiction)
Violet Blue — Smart Girl's Guide to Privacy (nonfiction)
Richard Caetano — Learning Bitcoin (nonfiction)
Nick Cano — Game Hacking (nonfiction)
Bruce Dang, et al. — Practical Reverse Engineering (nonfiction)
Chris Dannen — Introducing Ethereum and Solidity (nonfiction)
Daniel Drescher — Blockchain Basics (nonfiction)
Chris Eagle — The IDA Pro Book, 2nd Edition (nonfiction)
Nikolay Elenkov — Android Security Internals (nonfiction)
Jon Erickson — Hacking, 2nd Edition (nonfiction)
Pedro Franco — Understanding Bitcoin (nonfiction)
Christopher Hadnagy — Social Engineering (nonfiction)
Peter N.M. Hansteen — The Book of PF (nonfiction)
Brian Kelly — The Bitcoin Big Bang (nonfiction)
David Kennedy, et al. — Metasploit (nonfiction)
Manul Laphroaig (ed.) — PoC || GTFO (nonfiction)
Michael Hale Ligh, et al. — The Art of Memory Forensics
(nonfiction)
Michael Hale Ligh, et al. — Malware Analyst's Cookbook
(nonfiction)
Michael W. Lucas — Absolute OpenBSD, 2nd Edition (nonfiction)
Bruce Nikkel — Practical Forensic Imaging (nonfiction)
Sean-Philip Oriyano — CEHv9 (nonfiction)
Kevin D. Mitnick — The Art of Deception (nonfiction)
Narayan Prusty — Building Blockchain Projects (nonfiction)
Prypto — Bitcoin for Dummies (nonfiction)
Chris Sanders — Practical Packet Analysis, 3rd Edition
(nonfiction)
Bruce Schneier — Applied Cryptography (nonfiction)
Adam Shostack — Threat Modeling (nonfiction)
Craig Smith — The Car Hacker's Handbook (nonfiction)
Dafydd Stuttard & Marcus Pinto — The Web Application Hacker's
Handbook (nonfiction)
Albert Szmigielski — Bitcoin Essentials (nonfiction)
David Thiel — iOS Application Security (nonfiction)
Georgia Weidman — Penetration Testing (nonfiction)
Finally, the two SF bundles:
Buzz Aldrin & John Barnes — Encounter with Tiber (sff)
Poul Anderson — Orion Shall Rise (sff)
Greg Bear — The Forge of God (sff)
Octavia E. Butler — Dawn (sff)
William C. Dietz — Steelheart (sff)
J.L. Doty — A Choice of Treasons (sff)
Harlan Ellison — The City on the Edge of Forever (sff)
Toh Enjoe — Self-Reference ENGINE (sff)
David Feintuch — Midshipman's Hope (sff)
Alan Dean Foster — Icerigger (sff)
Alan Dean Foster — Mission to Moulokin (sff)
Alan Dean Foster — The Deluge Drivers (sff)
Taiyo Fujii — Orbital Cloud (sff)
Hideo Furukawa — Belka, Why Don't You Bark? (sff)
Haikasoru (ed.) — Saiensu Fikushon 2016 (sff anthology)
Joe Haldeman — All My Sins Remembered (sff)
Jyouji Hayashi — The Ouroboros Wave (sff)
Sergei Lukyanenko — The Genome (sff)
Chohei Kambayashi — Good Luck, Yukikaze (sff)
Chohei Kambayashi — Yukikaze (sff)
Sakyo Komatsu — Virus (sff)
Miyuki Miyabe — The Book of Heroes (sff)
Kazuki Sakuraba — Red Girls (sff)
Robert Silverberg — Across a Billion Years (sff)
Allen Steele — Orbital Decay (sff)
Bruce Sterling — Schismatrix Plus (sff)
Michael Swanwick — Vacuum Flowers (sff)
Yoshiki Tanaka — Legend of the Galactic Heroes, Volume 1: Dawn
(sff)
Yoshiki Tanaka — Legend of the Galactic Heroes, Volume 2: Ambition
(sff)
Yoshiki Tanaka — Legend of the Galactic Heroes, Volume 3: Endurance
(sff)
Tow Ubukata — Mardock Scramble (sff)
Sayuri Ueda — The Cage of Zeus (sff)
Sean Williams & Shane Dix — Echoes of Earth (sff)
Hiroshi Yamamoto — MM9 (sff)
Timothy Zahn — Blackcollar (sff)
Phew. Okay, all caught up, and hopefully won't have to dump something
like this again in the near future. Also, more books than I have any
actual time to read, but what else is new.
All posts