Eagle's Path

Passion and dispassion. Choose two.

Larry Wall

2018-07-30: Free software log (June 2018)

Well, this is embarassingly late, but not a full month late. That's what counts, right?

It's quite late partly because I haven't had the right combination of time and energy to do much free software work since the beginning of June. I did get a couple of releases out then, though. wallet 1.4 incorporated better Active Directory support and fixed a bunch of build system and configuration issues. And rra-c-util 7.2 includes a bunch of fixes to M4 macros and cleans up some test issues.

The July report may go missing for roughly the same reason. I have done some outside-of-work programming, but it's gone almost entirely into learning Rust and playing around with implementing various algorithms to understand them better. Rather fun, but not something that's good enough to be worth releasing. It's reinventing wheels intentionally to understand underlying concepts better.

I do have a new incremental release of DocKnot almost ready to go out (incorporating changes I needed for the last wallet release), but I'm not sure if that will make it into July.

2018-07-20: Review: The Power of Habit

Review: The Power of Habit, by Charles Duhigg

Publisher Random House
Copyright 2012, 2014
Printing 2014
ISBN 0-679-60385-9
Format Kindle
Pages 366

One problem with reading pop psychology is that one runs into a lot of books like this one: summaries of valid psychological research that still leave one with the impression that the author was more interested in being dramatic and memorable than accurate. But without reproducing the author's research, it's hard to tell whether that fear is well-grounded or unfair, so one comes away feeling vaguely dissatisfied and grumpy.

Or at least I do. I might be weird.

As readers of my book reviews may have noticed, and which will become more apparent shortly, I'm going through another round of reading "self-help" books. This time, I'm focusing on work habits, concentration, and how to more reliably reach a flow state. The Power of Habit isn't on that topic but it's adjacent to it, so I picked it up when a co-worker recommended it.

Duhigg's project here is to explain habits, both good ones and bad ones, at a scientific level. He starts with a memorable and useful model of the habit loop: a cue triggers a routine, which results in a reward. The reward reinforcement strengthens the loop, and the brain starts internalizing the routine, allowing it to spend less cognitive energy and essentially codifying the routine like a computer program. With fully-formed habits (one's daily bathing routine, for example), the routine is run by a small, tuned part of your brain and requires very little effort, which is why we can have profound shower thoughts about something else entirely. That example immediately shows why habits are valuable and why our brain is so good at creating them: they reduce the mental energy required for routine actions so that we can spend that energy elsewhere.

The problem, of course, is that this mechanism doesn't first consult our conscious intent. It works just as well for things that we do repeatedly but may not want to automatically do, like smoking a pack of cigarettes a day. It's also exploitable; you are not the only person involved in creating your habits. Essentially every consumer product company is trying to get you to form habits around their products, often quite successfully. Duhigg covers marketing-generated habits as well as social and societal habits, the science behind how habits can be changed, and the evidence that often a large collection of apparently unrelated habits are based in a "keystone habit" that, if changed, makes changing all of the other habits far easier.

Perhaps the most useful part of this book is Duhigg's discussion of how to break the habit loop through substitution. When trying to break habits, our natural tendency is to consciously resist the link between cue and routine. This is possible, but it's very hard. It requires making an unconscious process conscious, and we have a limited amount of conscious decision-making energy available to us in a day. More effective than fighting the cues is to build a replacement habit with the same cue, but this requires careful attention to the reward stage so that the substituted habit will complete the loop and have a chance of developing enough strength to displace the original habit.

So far, so good. All of this seems consistent with other psychological research I've read (particularly the reasons why trying to break habits by willpower alone is rarely successful). But there are three things that troubled me about this book and left me reluctant to recommend it or rely on it.

The first is that a useful proxy for checking the research of a book is to look at what the author says about a topic that one already knows something about. Here, I'm being a bit unfair by picking on a footnote, but Duhigg has one anecdote about a woman with a gambling problem that has following definitive-sounding note attached:

It may seem irrational for anyone to believe they can beat the house in a casino. However, as regular gamblers know, it is possible to consistently win, particularly at games such as blackjack. Don Johnson of Bensalem, Pennsylvania, for instance, won a reported $15.1 million at blackjack over a six-month span starting in 2010. The house always wins in the aggregate because so many gamblers bet in a manner that doesn't maximize their odds, and most people do not have enough money to see themselves through losses. A gambler can consistently win over time, though, if he or she has memorized the complicated formulas and odds that guide how each hand should be played. Most players, however, don't have the discipline or mathematical skills to beat the house.

This is just barely this side of being outright false, and is dangerously deceptive to the point of being casino propaganda. And the argument from anecdote is both intellectually bogus (a lot of people gamble, which means that not only is it possible that someone will go on that sort of winning streak through pure chance, it is almost guaranteed) and disturbingly similar to how most points are argued in this book.

If one assumes an effectively infinite deck (in other words, assume each card dealt is an independent event), there is no complicated rule you can memorize to beat the house at blackjack. The best that you can do is to reduce the house edge to 1-2% depending on the exact local rules. Wikipedia has a comprehensive discussion if you want the details. Therefore, what Duhigg has to be talking about is counting cards (modifying your play based on what cards have already been dealt and therefore what cards are remaining in the deck).

However, and Duhigg should know this if he's going to make definitive statements about blackjack, US casinos except in Atlantic City (every other example in this book is from the US) can and do simply eject players who count cards. (There's a legal decision affecting Atlantic City that makes the story more complicated there.) They also use other techniques (large numbers of decks, frequent reshuffling) to make counting cards far less effective. Even if you are very good at counting cards, this is not a way to win "consistently over time" because you will be told to stop playing. Counting cards is therefore not a matter of memorizing complicated formulas and odds. It's a cat-and-mouse game against human adversaries to disguise your technique enough to not be ejected while still maintaining an edge over the house. This is rather far from Duhigg's description.

Duhigg makes another, if less egregious, error by uncritically accepting the popular interpretation of the Stanford marshmallow experiment. I'll spare you my usual rant about this because The Atlantic has now written it for me. Surprise surprise, new research shows that the original experiment was deeply flawed in its choice of subjects and that the effect drastically decreases once one controls for social and economic background.

So that's one problem: when writing on topics about which I already have some background, he makes some significant errors. The second problem is related: Duhigg's own sources in this book seem unconvinced by the conclusions he's drawing from their research.

Here, I have to give credit to Duhigg for publishing his own criticism, although you won't find it if you read only the main text of the book. Duhigg has extensive end notes (distinct from the much smaller number of footnotes that elaborate on some point) in which he provides excerpts from fact-checking replies he got from the researchers and interview subjects in this book. I read them all after finishing the rest of the book, and I thought a clear pattern emerged. After reading early drafts of portions of the book, many of Duhigg's sources replied with various forms of "well, but." They would say that the research is accurately portrayed, but Duhigg's conclusion isn't justified by the research. Or that Duhigg described part of the research but left out other parts that complicated the picture. Or that Duhigg has simplified dangerously. Or that Duhigg latched on to an ancillary part of their research or their story and ignored the elements that they thought were more central. Note after note reads as a plea to add more nuance, more complication, less certainty, and fewer sweeping conclusions.

Science is messy. Psychological research is particularly messy because humans are very good at doing what they're "supposed" to do, or changing behavior based on subtle cues from the researcher. And most psychological research of the type Duhigg is summarizing is based on very small sample sizes (20-60 people is common) drawn from very unrepresentative populations (often college students who are conveniently near the researchers and cheap to bribe to do weird things while being recorded). When those experiments are redone with larger sample sizes or more representative populations, often they can't be replicated. This is called the replication crisis.

Duhigg is not a scientist. He's a reporter. His job is to take complicated and messy stories and simplify them into entertaining, memorable, and understandable narratives for a mass audience. This is great for making difficult psychological research more approachable, but it also inherently involves amplifying tentative research into rules of human behavior and compelling statements about how humans work. Sometimes this is justified by the current state of the research. Sometimes it isn't. Are Duhigg's core points in this book justified? I don't know and, based on the notes, neither does Duhigg, but none of that uncertainty is on the pages of the main text.

The third problem is less foundational, but seriously hurt my enjoyment of The Power of Habit as a reader: Duhigg's examples are horrific. The first chapter opens with the story of a man whose brain was seriously injured by a viral infection and could no longer form new memories. Later chapters feature a surgeon operating on the wrong side of a stroke victim's brain, a woman who destroyed her life and family through gambling, and a man who murdered his wife in his sleep believing she was an intruder. I grant that these examples are memorable, and some are part of a long psychological tradition of learning about the brain from very extreme examples, but these were not the images that I wanted in my head while reading a book about the science of habits. I'm not sure this topic should require the reader brace themselves against nightmares.

The habit loop, habit substitution, and keystone habits are useful concepts. Capitalist manipulation of your habits is something everyone should be aware of. There are parts of this book that seem worth knowing. But there's also a lot of uncritical glorification of particular companies and scientific sloppiness and dubious assertions in areas I know something about. I didn't feel like I could trust this book, or Duhigg. The pop psychology I like the best is either written by practicing scientists who (hopefully) have a feel for which conclusions are justified by research and which aren't, or admits more questioning and doubt, usually by personalizing the research and talking about what worked for the author. This is neither, and I therefore can't bring myself to recommend it.

Rating: 6 out of 10

2018-07-15: Review: Effective Python

Review: Effective Python, by Brett Slatkin

Publisher Addison-Wesley
Copyright 2015
ISBN 0-13-403428-7
Format Trade paperback
Pages 216

I'm still looking for a programming language book that's as good as Joshua Bloch's Effective Java, which goes beyond its surface mission to provide valuable and deep advice about how to think about software construction and interface design. Effective Python is, sadly, not that book. It settles for being a more pedestrian guide to useful or tricky corners of Python, with a bit of style guide attached (although not as much as I wanted).

Usually I read books like this as part of learning a language, but in this case I'd done some early experimenting with Python and have been using it extensively for my job for about the past four years. I was therefore already familiar with the basics and with some coding style rules, which made this book less useful. This is more of an intermediate than a beginner's book, but if you're familiar with list and hash comprehensions, closures, standard method decorators, context managers, and the global interpreter lock (about my level of experience when I started reading), at least half of this book will be obvious and familiar material.

The most useful part of the book for me was a deep look at Python's object system, including some fully-worked examples of mix-ins, metaclasses, and descriptors. This material was new to me and a bit different than the approach to similar problems in other programming languages I know. I think this is one of the most complex and hard-to-understand parts of Python and will probably use this as a reference the next time I have to deal with complex class machinery. (That said, this is also the part of Python that I think is the hardest to read and understand, so most programs are better off avoiding it.) The description of generators and coroutines was also excellent, and although the basic concepts will be familiar to most people who have done parallelism in other languages, Slatkin's treatment of parallelism and its (severe) limitations in Python was valuable.

But there were also a lot of things that I was surprised weren't covered. Some of these are due to the author deciding to limit the scope to the standard library, so testing only covers unittest and not the (IMO far more useful) pytest third-party module. Some are gaps in the language that the author can't fix (Python's documentation situation for user-written modules is sad). But there was essentially nothing here about distutils or how to publish modules properly, almost nothing about good namespace design and when to put code into __init__.py (a topic crying out for some opinionated recommendations), and an odd lack of mention of any static analysis or linting tools. Most books of this type I've read are noticeably more comprehensive and have a larger focus on how to share your code with others.

Slatkin doesn't even offer much of a style guide, which is usually standard in a book of this sort. He does steer the reader away from a few features (such as else with for loops) and preaches the merits of decomposition and small functions, among other useful tidbits. But it falls well short of Damian Conway's excellent guide for Perl, Perl Best Practices.

Anyone who already knows Python will be wondering how Slatkin handles the conflict between Python 2 and Python 3. The answer is that it mostly doesn't matter, since Slatkin spends little time on the parts of the language that differ. In the few places it matters, Effective Python discusses Python 3 first and then mentions the differences or gaps in Python 2. But there's no general discussion about differences between Python 2 and 3, nor is there any guide to updating your own programs or attempting to be compatible with both versions. That's one of the more common real-world problems in Python at the moment, and was even more so when this book was originally written, so it's an odd omission.

Addison-Wesley did a good job on the printing, including a nice, subtle use of color that made the physical book enjoyable to read. But the downside is that this book has a surprisingly expensive retail ($40 USD) for a fairly thin trade paperback. At the time of this writing, Amazon has it on sale at 64% off, which takes the cost down to about the right territory for what you get.

I'm not sorry I read this, and I learned a few things from it despite having used Python fairly steadily for the last few years. But it's nowhere near good enough to recommend to every Python programmer, and with a bit of willingness to search out on-line articles and study high-quality code bases, you can skip this book entirely and never miss it. I found it oddly unopinionated and unsatisfying in the places where I wish Python had more structure or stronger conventions. This is particularly strange given that it was written by a Google staff engineer and Google has a quite comprehensive and far more opinionated coding style guide for Python.

If you want to dig into some of Python's class and object features or see a detailed example of how to effectively use coroutines, Effective Python is a useful guide. Otherwise, you'll probably learn some things from this book, but it's not going to significantly change how you approach the language.

Rating: 6 out of 10

2018-06-23: Review: The Trouble with Physics

Review: The Trouble with Physics, by Lee Smolin

Publisher Mariner
Copyright 2006
Printing 2007
ISBN 0-618-91868-X
Format Trade paperback
Pages 355

A brief recap of the state of theoretical physics: Quantum mechanics and particle physics have settled on the standard model, which provides an apparently complete inventory of fundamental particles and explains three of the four fundamental forces. This has been very experimentally successful up to and including the recent tentative observation of the Higgs boson, one of the few predictions of the standard model that had yet to be confirmed by experiment. Meanwhile, Einstein's theory of general relativity continues as the accepted explanation of gravity, experimentally verified once again by LIGO and Virgo detection of gravitational waves.

However, there are problems. Perhaps the largest is the independence of these two branches of theoretical physics: quantum mechanics does not include or explain gravity, and general relativity does not sit easily alongside current quantum theory. This causes theoretical understanding to break down in situations where both theories need to be in play simultaneously, such as the very early universe or event horizons of black holes.

There are other problems within both theories as well. Astronomy shows that objects in the universe behave as if there is considerably more mass in galaxies than we've been able to observe (the dark matter problem), but we don't have a satisfying theory of what would make up that mass. Worse, the universe is expanding more rapidly than it should, requiring introduction of a "dark energy" concept with no good theoretical basis. And, on the particle physics side, the standard model requires a large number (around 20, depending on how you measure them) of apparently arbitrary free constants: numbers whose values don't appear to be predicted by any basic laws and therefore could theoretically be set to any value. Worse, if those values are set even very slightly differently than we observe in our universe, the nature of the universe would change beyond recognition. This is an extremely unsatisfying property for an apparently fundamental theory of nature.

Enter string theory, which is the dominant candidate for a deeper, unifying theory behind the standard model and general relativity that tries to account for at least some of these problems. And enter this book, which is a critique of string theory as both a scientific theory and a sociological force within the theoretical physics community.

I should admit up-front that Smolin's goal in writing this book is not the same as my goal in reading it. His primary concern is the hold that string theory has on theoretical physics and the possibility that it is stifling other productive avenues, instead spinning off more and more untestable theories that can be tweaked to explain any experimental result. It may even be leading people to argue against the principles of experimental science itself (more on that in a moment). But to mount his critique for the lay reader, he has to explain the foundations of both accepted theoretical physics and string theory (and a few of the competing alternative theories). That's what I was here for.

About a third of this book is a solid explanation of the history and current problems of theoretical physics for the lay person who is already familiar with basic quantum mechanics and general relativity. Smolin is a faculty member at the Perimeter Institution for Theoretical Physics and has done significant work in string theory, loop quantum gravity (one of the competing attempts to unify quantum mechanics and general relativity), and the (looking dubious) theory of doubly special relativity, so this is an engaged and opinionated overview from an active practitioner. He lays out the gaps in existing theories quite clearly, conveys some of the excitement and disappointment of recent (well, as of 2005) discoveries and unsolved problems, provides a solid if succinct summary of string theory, and manages all of that without relying on too much complex math. This is exactly the sort of thing I was looking for after Brian Greene's The Elegant Universe.

Another third of this book is a detailed critique of string theory, and specifically the assumption that string theory is correct despite its lack of testable predictions and its introduction of new problems. I noted in my review of Greene's book that I was baffled by his embrace of a theory that appears to add even more free variables than the standard model, an objection that he skipped over entirely. Smolin tackles this head-on, along with other troublesome aspects of a theory that is actually an almost infinitely flexible family of theories and whose theorized unification (M-theory) is still just an outline of a hoped-for idea.

The core of Smolin's technical objection to string theory is that it is background-dependent. Like quantum mechanics, it assumes a static space-time backdrop against which particle or string interactions happen. However, general relativity is background-independent; indeed, that's at the core of its theoretical beauty. It states that the shape of space-time itself changes, and is a participant in the physical effects we observe (such as gravity). Smolin argues passionately that background independence is a core requirement for any theory that aims to unify general relativity and quantum mechanics. As long as a theory remains background-dependent, it is, in his view, missing Einstein's key insight.

The core of his sociological objection is that he believes string theory has lost its grounding in experimental verification and has acquired far too much aura of certainty than it deserves given its current state, and has done so partly because of the mundane but pernicious effects of academic and research politics. On this topic, I don't know nearly enough to referee the debate, but his firm dismissal of attempts to justify string theory's weaknesses via the anthropic principle rings true to me. (The anthropic principle, briefly, is the idea that the large number of finely-tuned free constants in theories of physics need not indicate a shortcoming in the theory, but may be that way simply because, if they weren't, we wouldn't be here to observe them.) Smolin's argument is that no other great breakthroughs of physics have had to rely on that type of hand-waving, elegance of a theory isn't sufficient justification to reach for this sort of defense, and that to embrace the anthropic principle and its inherent non-refutability is to turn one's back on the practice of science. I suspect this ruffled some feathers, but Smolin put his finger squarely on the discomfort I feel whenever the anthropic principle comes up in scientific discussions.

The rest of the book lays out some alternatives to string theory and some interesting lines of investigation that, as Smolin puts it, may not pan out but at least are doing real science with falsifiable predictions. This is the place where the book shows its age, and where I frequently needed to do some fast Wikipedia searching. Most of the experiments Smolin points out have proven to be dead ends: we haven't found Lorentz violations, the Pioneer anomaly had an interesting but mundane explanation, and the predictions of modified Newtonian dynamics do not appear to be panning out. But I doubt this would trouble Smolin; as he says in the book, the key to physics for him is to make bold predictions that will often be proven wrong, but that can be experimentally tested one way or another. Most of them will lead to nothing but one can reach a definitive result, unlike theories with so many tunable parameters that all of their observable effects can be hidden.

Despite not having quite the focus I was looking for, I thoroughly enjoyed this book and only wish it were more recent. The physics was pitched at almost exactly the level I wanted. The sociology of theoretical physics was unexpected but fascinating in a different way, although I'm taking it with a grain of salt until I read some opposing views. It's an odd mix of topics, so I'm not sure if it's what any other reader would be looking for, but hopefully I've given enough of an outline above for you to know if you'd be interested.

I'm still looking for the modern sequel to One Two Three... Infinity, and I suspect I may be for my entire life. It's hard to find good popularizations of theoretical physics that aren't just more examples of watching people bounce balls on trains or stand on trampolines with bowling balls. This isn't exactly that, but it's a piece of it, and I'm glad I read it. And I wish Smolin the best of luck in his quest for falsifiable theories and doable experiments.

Rating: 8 out of 10

2018-06-04: Review: The Obelisk Gate

Review: The Obelisk Gate, by N.K. Jemisin

Series The Broken Earth #2
Publisher Orbit
Copyright August 2016
ISBN 0-316-22928-8
Format Kindle
Pages 448

The Obelisk Gate is the sequel to The Fifth Season and picks up right where it left off. This is not a series to read out of order.

The complexity of The Fifth Season's three entwined stories narrows down to two here, which stay mostly distinct. One follows Essun, who found at least a temporary refuge at the end of the previous book and now is split between learning a new community and learning more about the nature of the world and orogeny. The second follows Essun's daughter, whose fate had been left a mystery in the first book. This is the middle book of a trilogy, and it's arguably less packed with major events than the first book, but the echoing ramifications of those events are vast and provide plenty to fill a novel. The Obelisk Gate never felt slow. The space between major events is filled with emotional processing and revelations about the (excellent) underlying world-building.

We do finally learn at least something about the stone-eaters, although many of the details remain murky. We also learn something about Alabaster's goals, which were the constant but mysterious undercurrent of the first book. Mixed with this is the nature of the Guardians (still not quite explicit, but much clearer now than before), the purpose of the obelisks, something of the history that made this world such a hostile place, and the underlying nature of orogeny.

The last might be a touch disappointing to some readers (I admit it was a touch disappointing to me). There are enough glimmers of forgotten technology and alternative explanations that I was wondering if Jemisin was setting up a quasi-technological explanation for orogeny. This book makes it firmly clear that she's not: this is a fantasy, and it involves magic. I have a soft spot in my heart for apparent magic that's some form of technology, so I was a bit sad, but I do appreciate the clarity. The Obelisk Gate is far more open with details and underlying systems (largely because Essun is learning more), which provides a lot of meat for the reader to dig into and understand. And it remains a magitech world that creates artifacts with that magic and uses them (or, more accurately, used them) to build advanced civilizations. I still see some potential pitfalls for the third book, depending on how Jemisin reconciles this background with one quasi-spiritual force she's introduced, but the world building has been so good that I have high hopes those pitfalls will be avoided.

The world-building is not the best part of this book, though. That's the characters, and specifically the characters' emotions. Jemisin manages the feat of both giving protagonists enough agency that the story doesn't feel helpless while still capturing the submerged rage and cautious suspicion that develops when the world is not on your side. As with the first book of this series, Jemisin captures the nuances, variations, and consequences of anger in a way that makes most of fiction feel shallow.

I realized, while reading this book, that so many action-oriented and plot-driven novels show anger in only two ways, which I'll call "HULK SMASH!" and "dark side" anger. The first is the righteous anger when the protagonist has finally had enough, taps some heretofore unknown reservoir of power, and brings the hurt to people who greatly deserved it. The second is the Star Wars cliche: anger that leads to hate and suffering, which the protagonist has to learn to control and the villain gives into. I hadn't realized how rarely one sees any other type of anger until Jemisin so vividly showed me the vast range of human reaction that this dichotomy leaves out.

The most obvious missing piece is that both of those modes of anger are active and empowered. Both are the anger of someone who can change the world. The argument between them is whether anger changes the world in a good way or a bad way, but the ability of the angry person to act on that anger and for that anger to be respected in some way by the world is left unquestioned. One might, rarely, see helpless anger, but it's usually just the build-up to a "HULK SMASH!" moment (or, sometimes, leads to a depressing sort of futility that makes me not want to read the book at all).

The Obelisk Gate felt like a vast opening-up of emotional depth that has a more complicated relationship to power: hard-earned bitterness that brings necessary caution, angry cynicism that's sometimes wrong but sometimes right, controlled anger, anger redirected as energy into other actions, anger that flares and subsides but doesn't disappear. Anger that one has to live with, and work around, and understand, instead of getting an easy catharsis. Anger with tradeoffs and sacrifices that the character makes consciously, affected by emotion but not driven by it. There is a moment in this book where one character experiences anger as an overwhelming wave of tiredness, a sharp realization that they're just so utterly done with being angry all the time, where the emotion suddenly shifts into something more introspective. It was a beautifully-captured moment of character depth that I don't remember seeing in another book.

This may sound like it would be depressing and exhausting to read, but at least for me it wasn't at all. I didn't feel like I was drowning in negative emotions — largely, I think, because Jemisin is so good at giving her characters agency without having the world give it to them by default. The protagonists are self-aware. They know what they're angry about, they know when anger can be useful and when it isn't, and they know how to guide it and live with it. It feels more empowering because it has to be fought for, carved out of a hostile world, earned with knowledge and practice and stubborn determination. Particularly in Essun, Jemisin is writing an adult whose life is full of joys and miseries, who doesn't forget her emotions but also isn't controlled by them, and who doesn't have the luxury of either being swept away by anger or reaching some zen state of unperturbed calm.

I think one key to how Jemisin pulls this off is the second-person perspective used for Essun's part of the book (and carried over into the other strand, which has the same narrator but a different perspective since this story is being told to Essun). That's another surprise, since normally this style strikes me as affected and artificial, but here it serves the vital purpose of giving the reader a bit of additional distance from Essun's emotions. Following an emotionally calmer retelling of someone else's perspective on Essun made it easier to admire what Jemisin is doing with the nuances of anger without getting too caught up in it.

It helps considerably that the second-person perspective here has a solid in-story justification (not explicitly explained here, but reasonably obvious by the end of the book), and is not simply a gimmick. The answers to who is telling this story and why they're telling it to a protagonist inside the story are important, intriguing, and relevant.

This series is doing something very special, and I'm glad I stuck to it through the confusing and difficult parts in the first book. There's a reason why every book in it was nominated for the Hugo and The Obelisk Gate won in 2017 (and The Fifth Season in 2016). Despite being the middle book of a trilogy, and therefore still leaving unresolved questions, this book was even better than The Fifth Season, which already set a high bar. This is very skillful and very original work and well worth the investment of time (and emotion).

Followed by The Stone Sky.

Rating: 9 out of 10

2018-06-03: rra-c-util 7.2

rra-c-util is my collection of portability and utility code that I reuse in all the C and Perl projects I maintain.

Most of the changes in this release are Autoconf macro improvements prompted by Julien ÉLIE. This release incorporates his work on RRA_PROG_PYTHON, RRA_LIB_PYTHON, and RRA_PYTHON_MODULE macros, which support both Python 2 and Python 3. It also improves the RRA_PROG_PERL macro to make PERL a substitution variable and to check that building against libperl actually works. Finally, RRA_LIB_BDB, RRA_LIB_OPENSSL, RRA_LIB_SASL, and RRA_LIB_ZLIB now check that the headers for the library are found as well as the library itself (based on Julien's work in INN).

The docs/urls test, which was always misnamed, is now style/obsolete-strings, since its role is to check for obsolete patterns in my code (old URLs, that sort of thing). It now checks for my old RRA_MAINTAINER_TESTS environment variable, which I replaced with the Perl Lancaster Consensus environment variables a long time ago.

This release also fixes a few more minor issues with test code and the script to update the version of all Perl modules in a package.

You can get the latest release from the rra-c-util distribution page.

2018-06-03: wallet 1.4

wallet is a secret management system that I developed at Stanford, primarily to distribute keytab management. As mentioned in an earlier post, I'm not entirely sure it has significant advantages over Vault, but it does handle Kerberos natively and we're still using it for some things, so I'm still maintaining it.

This release incorporates a bunch of improvements to the experimental support for managing keytabs for Active Directory principals, all contributed by Bill MacAllister and Dropbox. Anyone using the previous experimental Active Directory support should read through the configuration options, since quite a lot has changed (for the better).

Also fixed in this release are some stray strlcpy and strlcat references that were breaking systems that include them in libc, better krb5.conf configuration handling, better support for Perl in non-standard locations, and a bunch of updates and modernization to the build and test frameworks.

You can get the latest release from the wallet distribution page.

2018-06-03: Free software log (May 2018)

The wonders of a week of vacation that was spent mostly working on free software! The headline releases were remctl 3.15, which fixes a long-standing correctness bug on the server and adds more protocol validation and far better valgrind support, and podlators 4.11, which fixes a buncho f long-standing bugs in Pod::Text and its subclasses.

In support of those releases, I also released new versions of my three major development infrastructure packages:

On the Debian front, I realized that I had intended to donate libnet-duo-perl to the Debian Perl team but never finished uploading the package I had prepared (and even signed). I merged that with some other pending changes in Git and actually uploaded it. (I'm still hanging on to maintenance of the upstream Net::Duo Perl module because I'm kicking around the idea of using Duo on a small scale for some personal stuff, although at the moment I'm not using the module at all and therefore am not making changes to it.)

I also finally started working on wallet again, although I'm of two minds about the future of that package. It needs a ton of work — the Perl style and general backend approach is all wrong, and I've learned far better ways to do equivalent things since. And one could make a pretty solid argument that Vault does essentially the same thing, has a lot more resources behind it, and has a ton of features that I haven't implemented or may never implement. I think I still like my ACL model better, and of course there's the Kerberos support (which is probably superior to Vault), but I haven't looked at Vault closely enough to be sure and it may be that it's better in those areas as well.

I don't use wallet for my personal stuff, but we still do use it in a few places at work. I kind of want to overhaul the package and fix it, since I like the concept, but in the broader scheme of things it's probably a "waste" of my time to do this.

Free software seems full of challenges like this. I'll at least put out another release, and then probably defer making a decision for a while longer.

2018-05-29: Review: Bull by the Horns

Review: Bull by the Horns, by Sheila Bair

Publisher Simon & Schuster
Copyright 2012
Printing September 2013
ISBN 1-4516-7249-7
Format Trade paperback
Pages 365

Sheila Bair was the Chair of the Federal Deposit Insurance Corporation from 2006 to 2011, a period that spans the heart of the US housing crisis and the start of the Great Recession. This is her account, based on personal notes, of her experience heading the FDIC, particularly focused on the financial crisis and its immediate aftermath.

Something I would like to do in theory but rarely manage to do in practice is to read more thoughtful political writing from people who disagree with me. Partly that's to broaden my intellectual horizons; partly it's a useful reminder that the current polarized political climate in the United States does not imply that the intellectual tradition of conservatism is devoid of merit. While it's not a complete solution, one way to edge up on such reading is to read books by conservatives that are focused on topics where they and I largely agree.

In this case, that topic is the appalling spectacle of consequence-free government bailouts of incompetently-run financial institutions, coordinated by their co-conspirators inside the federal government and designed to ensure that obscenely large salaries and bonuses continued to flow to exactly the people most responsible for the financial crisis. If I sound a little heated on this topic, well, consider it advance warning for the rest of the review. Suffice it to say that I consider Timothy Geithner to be one of the worst Secretaries of the Treasury in the history of the United States, a position for which the competition is fierce.

Some background on the US financial regulatory system might be helpful here. I'm reasonably well-read on this topic and still learned more about some of the subtleties.

The FDIC, which Bair headed, provides deposit insurance to all of the banks. This ensures that whatever happens to the bank, all depositors of up to $100,000 (now $250,000 due to a law that was passed as part of the events of this book) are guaranteed to get every cent of their money. This deposit insurance is funded by fees charged to every bank, not by general taxes, although the FDIC has an emergency line of credit with the Treasury it can call on (and had to during the savings and loan crisis in the early 1990s).

The FDIC is also the primary federal regulator for state banks. It is not the regulator for federal banks; those are regulated by the Office of the Comptroller of the Currency (OCC) and, at the time of events in this book, the Office of Thrift Supervision (OTS), which regulated Savings and Loans. Some additional regulation of federal banks is done by the Federal Reserve. The FDIC is a "backup" regulator to those other institutions and has some special powers related to its function of providing deposit insurance, but it doesn't in general have the power to demand changes of federal banks, only the smaller state banks.

This turns out to be rather important in the financial crisis: bad state banks regulated by the FDIC were sold off or closed, but the huge federal banks regulated by the OCC and OTS were bailed out via various arranged mergers, loan guarantees, or direct infusions of taxpayer money. Bair's argument is that this difference is partly due to the ethos of the FDIC and its well-developed process for closing troubled banks. The standard counter-argument is that the large national banks were far too large to put through that or some similar process without massive damage to the economy. (Bair strenuously disagrees.)

Bair's account starts in 2006, by which point the crisis was already probably inevitable, and contains a wealth of information about the banking side of the crisis itself and its immediate aftermath. Her story is one of consistent pressure by the FDIC to increase bank capital requirements and downgrade risk ratings of institutions, and consistent pressure by the OCC, OTS, and Geithner (first as the head of the New York branch of the Federal Reserve and then as Treasury Secretary) to decrease capital requirements even in the height of the crisis and allow banks to use ever-more-creative funding models backed by government guarantees. Bair fleshes this out with considerable detail about how capital requirements are measured, how the loan guarantees were structured, the internal arguments over how to get control of the crisis, and the subsequent fights in Congress over Dodd-Frank and how TARP money was spent.

(TARP, the Troubled Asset Relief Program, was the Congressional emergency measure passed during the height of the crisis to fund government purchases and restructuring of troubled mortgage debt. As Bair describes, and has been exhaustively detailed elsewhere, it was never really used for that. The government almost immediately repurposed it for direct bailouts of financial institutions and provided almost no meaningful mortgage restructuring.)

This account also passes my primary sniff test for books about this crisis. Fannie and Freddie (two oddly-named US government institutions with a mandate to support mortgage lending and home ownership) are treated as bad actors and horribly mismanaged entities that made the same irresponsible investments as the private banking industry, but they aren't put at the center of the crisis and aren't blamed for the entire mortgage mess. This disagrees with some corners of Republican politics, but agrees with all other high-quality reporting about the crisis.

Besides fascinating details about the details of banking regulation in a crisis, the primary conclusion I drew from this book is the power of institutions, systems, and rules. One becomes good at things one does regularly. The FDIC closes failing banks without losing insured depositor money, and has been doing that since 1933, often multiple times a year. They therefore have a tested system for doing this, which they practice implementing reliably, efficiently, and quickly. Bair states as a point of deep institutional pride that no insured depositor had to wait more than one business day for access to their funds during the financial crisis. Banks are closed after business hours and, whenever possible, the branches was open for business under new supervision the next morning. This is as important as the insurance in preventing runs on the bank that would make the closing cost even more.

Part of that system, built into the FDIC principles and ethos, was a ranking of priorities and a deep sense of the importance of consequences. Insured depositors are sacrosanct. Uninsured depositors are not, but often they can be protected by selling the bank assets to another, healthier bank, since the uninsured depositors are often the bank's best customers. Investors in the bank, in contrast, are wiped out. And other creditors may also be wiped out, or at least have to take a significant haircut on their investment. That is the price of investing in a failed institution; next time, pay more attention to the health of the business you're investing in. The FDIC is legally required to choose the resolution approach that is the least costly to the deposit insurance fund, without regard to the impact on the bank's other creditors.

And, finally, when the FDIC takes over a failing bank, one of the first things they do is fire all of the bank management. Bair presents this as obvious and straight-forward common sense, as it should be. These were the people who created the problem. Why would you want to let them continue to mismanage the bank? The FDIC may retain essential personnel needed to continue bank operations, but otherwise gets rid of the people who should bear direct responsibility for the bank's failure.

The contrast with the government's approach with AIG, Citigroup, and other failed financial institutions, as spearheaded by Timothy Geithner, could not be more stark. I remember following the news at the time and seeing straight-faced and serious statements that it was important to preserve the compensation and bonuses of the CEOs of failed institutions so that they would continue to work for the institution to unwind all of its bad trades and troubled assets. Bair describes herself as furious over that decision.

The difficulty in critiques of the government's approach to the financial crisis has always been that it was a crisis, with unknown possible consequences, and the size of the shadow banking sector and the level of entangled risk was so large that any systematic bankruptcy process would have been too risky. I'm with Bair in finding this argument dubious but not clearly incorrect. The Lehman Brothers bankruptcy was rocky, but it's not clear to me that a similar process couldn't have worked for other firms. But that aside, retaining the corporate management (and their salaries and bonuses!) seems a clear indication to me of the corruption of the system. (Bair, possibly more to her credit than mine, carefully avoids using that term.)

Bair highlights this as one of the critical reasons why the FDIC process is legally akin to bankruptcy: these sorts of executives write themselves sweetheart employment contracts that guarantee huge payouts even if their company fails. In the FDIC resolution process, those contracts can be broken. If, as Geithner did, you take heroic measures to avoid going anywhere near bankruptcy law, breaking those contracts becomes more legally murky. (Dodd-Frank has a provision, strongly supported by Bair, to create a legal framework for clawing back compensation to executives after certain types of financial misreporting, although it's still far more limited than the FDIC resolution process.)

A note of caution here: this book is obviously Bair's personal account, and she's not an unbiased party. She took specific public positions during the crisis and defends them here, including against analysis in other books about the crisis. She also describes lots of private positions, some of which are disputed. (Andrew Ross Sorkin's book is the subject of some particularly pointed disagreement.) I have read enough other books about the crisis to believe that Bair's account is probably essentially correct, particularly given the nature of the contemporaneous criticism against her. But, that said, the public position against bailouts had become quite clear by the time she was writing this book, and there was doubtless some temptation to remember her previous positions as more in line with later public opinion than they were. This sort of insider account is always worth a note of caution and some effort to balance it with other accounts, particularly given Bair's love of the spotlight (which shines through in a few places in this book).

Bair is a life-long Republican and a Bush appointee. I suspect she and I would disagree on most political positions. But her position as head of the FDIC was that bank failure should come with consequences for those running the bank, that the priority of the government should be protection of insured bank depositors first and the deposit insurance fund second, and that other creditors should bear the brunt of their bad investment decisions, all of which I agree with wholeheartedly. This account is an argument for the importance of moral hazard, and an indictment and diagnosis of regulatory capture from someone who (refreshingly) is not just using that as a stalking horse to argue for eliminating regulation. Bair also directly tackles the question of whether the same moral hazard argument applies to the individual loan holders and concludes no, but this part of the argument was a bit light on detail and probably won't convince someone with the opposite opinion.

It's quite frustrating, reading this in 2018, how many of the reforms Bair argues for in this book never happened. (A ban on naked credit default swaps, for example, which Bair argues increase systemic risk by increasing the consequences of institutional bankruptcy, thus creating new "too big to fail" analyses like that applied to AIG. Timothy Geithner was central to defeating an effort to outlaw them.) It's also a tragic reminder of how blindly partisan our national debates over economic policies are. You can watch, in Bair's account, the way that Democrats who were sharply critical of the Bush administration handling of the financial crisis, including his appointed regulators, swung behind the exact same regulators and essentially the same policies when Obama appointed Geithner to head Treasury. Democrats are traditionally the party favoring stronger regulation, but that's less important than tribal affiliation. The change is sharp enough that at a few points I was caught by surprise at the political affiliation of a member of Congress who was supporting or opposing one of Bair's positions.

As infuriating as this book is in places, it is a strong reminder that there are conservatives with whom I can find common cause despite being on the hard left of US economic politics. Those tend to be the people who believe in the power of institutions, consistent principles, and repeated and efficient execution of processes developed through hard-fought political compromise. I think Bair and I would agree that it's very dangerous to start making up policies on the spot to deal with the crisis du jour. Corruption can more easily enter the system, and very bad decisions are made. This is a failure on both the left and the right. I suspect Bair would turn to a principle of smaller government far more than I would, but we both believe in better government and clear, principled regulation, and on that point we could easily find workable compromises.

You should not read this as your first in-depth look at the US financial crisis. For that, I still recommend McLean & Nocera's All the Devils are Here. But this is a good third or fourth book on the topic, and a deep look at the internal politics around TARP. If that interests you, recommended.

Rating: 8 out of 10

2018-05-27: Slightly excessive haul

Okay, yes, I've been buying a lot of books. I seem to be ensuring the rate of acquisition outpaces the rate of consumption even though I've gotten back into reading daily on the train. Well, also, Diane Duane had a crisis and therefore a book sale, so I picked up a whole pile of stuff that I don't have an immediate intention to read.

I still have three written but not yet posted reviews to go through, which are waiting on me finding time to do some editing, and another finished book to review. I'm finishing more non-fiction lately than fiction, possibly due to a less-than-ideal choice of the fiction book for my mood.

The problem with non-fiction is that non-fiction authors keep recommending other interesting-sounding books to read!

Ilona Andrews — On the Edge (sff)
Catherine Asaro (ed.) — Irresistible Forces (sff anthology)
Isaac Asimov (ed.) — The New Hugo Winners (sff anthology)
Fredrik Backman — Beartown (mainstream)
Steven Brust — Good Guys (sff)
Steven Brust & Skyler White — The Skill of Our Hands (sff)
Jo Clayton — Skeen's Leap (sff)
Jo Clayton — Skeen's Return (sff)
Jo Clayton — Skeen's Search (sff)
Diane Duane — So You Want to Be a Wizard (sff)
Diane Duane — Deep Wizardry (sff)
Diane Duane — High Wizardry (sff)
Diane Duane — A Wizard Abroad (sff)
Diane Duane — The Wizard's Dilemma (sff)
Diane Duane — A Wizard Alone (sff)
Diane Duane — Wizard's Holiday (sff)
Diane Duane — Wizards at War (sff)
Diane Duane — A Wizard of Mars (sff)
Diane Duane — Tale of the Five (sff)
Diane Duane — The Big Meow (sff)
Charles Duhigg — The Power of Habit (nonfiction)
Max Gladstone — Four Roads Cross (sff)
Max Gladstone — The Ruin of Angels (sff)
Alison Green — Ask a Manager (nonfiction)
Nicola Griffith — So Lucky (mainstream)
Dorothy J. Heydt — The Witch of Syracuse (sff)
N.K. Jemisin — The Awakened Kingdom (sff)
Richard Kadrey — From Myst to Riven (nonfiction)
T. Kingfisher — The Wonder Engine (sff)
Ilana C. Myer — Last Song Before Night (sff)
Cal Newport — Deep Work (nonfiction)
Cal Newport — So Good They Can't Ignore You (nonfiction)
Emilie Richards — When We Were Sisters (mainstream)
Graydon Saunders — The Human Dress (sff)
Bruce Schneier — Data and Goliath (nonfiction)
Brigid Schulte — Overwhelmed (nonfiction)
Rivers Solomon — An Unkindness of Ghosts (sff)
Douglas Stone & Sheila Heen — Thanks for the Feedback (nonfiction)
Jodi Taylor — Just One Damned Thing After Another (sff)
Catherynne M. Valente — Space Opera (sff)


You'll notice a few in here that I've already read and reviewed.

The anthologies, the Backman, and a few others are physical books my parents were getting rid of.

So much good stuff in there I really want to read! And of course I've now started various other personal projects that don't involve spending all of my evenings and weekends reading.

2018-05-19: California state election

Hm, I haven't done one of these in a while. Well, time to alienate future employers and make awkward mistakes in public that I have to explain if I ever run for office! (Spoiler: I'm highly unlikely to ever run for office.)

This is only of direct interest to California residents. To everyone else, RIP your feed reader, and I'm sorry for the length. (My hand-rolled blog software doesn't do cut tags.) I'll spare you all the drill-down into the Bay Area regional offices. (Apparently we elect our coroner, which makes no sense to me.)


I'm not explaining these because this is already much too long; those who aren't in California and want to follow along can see the voter guide.

Proposition 68: YES. Still a good time to borrow money, and what we're borrowing money for here seems pretty reasonable. State finances are in reasonable shape; we have the largest debt of any state for the obvious reason that we have the most people and the most money.

Proposition 69: YES. My instinct is to vote no because I have a general objection to putting restrictions on how the state manages its budget. I don't like dividing tax money into locked pools for the same reason that I stopped partitioning hard drives. That said, this includes public transit in the spending pool from gasoline taxes (good), the opposition is incoherent, and there are wide-ranging endorsements. That pushed me to yes on the grounds that maybe all these people understand something about budget allocations that I don't.

Proposition 70: NO. This is some sort of compromise with Republicans because they don't like what cap-and-trade money is being spent on (like high-speed rail) and want a say. If I wanted them to have a say, I'd vote for them. There's a reason why they have to resort to backroom tricks to try to get leverage over laws in this state, and it's not because they have good ideas.

Proposition 71: YES. Entirely reasonable change to say that propositions only go into effect after the election results are final. (There was a real proposition where this almost caused a ton of confusion, and prompted this amendment.)

Proposition 72: YES. I'm grumbling about this because I think we should get rid of all this special-case bullshit in property taxes and just readjust them regularly. Unfortunately, in our current property tax regime, you have to add more exemptions like this because otherwise the property tax hit (that would otherwise not be incurred) is so large that it kills the market for these improvements. Rainwater capture is to the public benefit in multiple ways, so I'll hold my nose and vote for another special exception.

Federal Offices

US Senator: Kevin de León. I'll vote for Feinstein in the general against any Republican, and she's way up on de León in the polls, but there's no risk in voting for the more progressive candidate here since there's no chance Feinstein won't get the most votes in the primary. De León is a more solidly progressive candidate than Feinstein. I'd love to see a general election between the two of them.

State Offices

I'm omitting all the unopposed ones, and all the ones where there's only one Democrat running in the primary. (I'm not going to vote for any Republican except for one exception noted below, and third parties in the US are unbelievably dysfunctional and not ready to govern.) For those outside the state, California has a jungle primary where the top two vote-getters regardless of party go to the general election, so this is more partisan and more important than other state primaries.

Governor: Delaine Eastin. One always has to ask, in our bullshit voting system, whether one has to vote tactically instead of for the best candidate. But, looking at polling, I think there's no chance Gavin Newsom (the second-best candidate and the front-runner) won't advance to the general election, so I get to vote for the candidate I actually want to win, even though she's probably not going to. Eastin is by far the most progressive candidate running who actually has the experience required to be governor. (Spoiler: Newsom is going to win, and I'll definitely vote for him in the general against Villaraigosa.)

Lieutenant Governor: Eleni Kounalakis. She and Bleich are the strongest candidates. I don't see a ton of separation between them, but Kounalakis's endorsements are a bit stronger for me. She's also the one candidate who has a specific statement about what she plans to do with the lieutenant governor role of oversight over the university system, which is almost its only actual power. (This political office is stupid and we should abolish it.)

Secretary of State: Alex Padilla. I agree more with Ruben Major's platform (100% paper ballots is the correct security position), but he's an oddball outsider and I don't think he can accomplish as much. Padilla has an excellent track record as the incumbant and is doing basically the right things, just less dramatically.

Treasurer: Fiona Ma. I like Vivek Viswanathan and support his platform, but Fiona Ma has a lot more political expertise and I think will be more effective. I look forward to voting for Viswanathan for something else someday.

Attorney General: Dave Jones. Xavier Becerra hasn't been doing a bad job fighting off bad federal policy, but that seems to be all that he's interested in, and he's playing partisan games with the office. He has an air of amateurishness and political hackery. Dave Jones holds the same positions in a more effective way, is more professional, and has done a good job as Insurance Commissioner.

Insurance Commissioner: Steve Poizner. I'm going to vote for the (former) Republican here. Poizner expressed some really bullshit views on immigration when he ran for governor (which he's apologized for). I wouldn't support him for a more political office. But he was an excellent insurance commissioner (see, for instance, the response to Blue Cross's rate increase request). I'm closer to Ricardo Lara politically, but in his statements to the press he comes across as a clown: self-driving car insurance problems, cannabis insurance, climate change insurance, and a bunch of other nonsense that makes me think he doesn't understand the job. The other democrat, Mahmood, seems like less of a partisan hack, but he's a virtual unknown. If this were an important partisan office, I'd hold my nose and vote for Lara, but the job of insurance commissioner is more to be an auditor and negotiator, and Poizner was really good at it.

Superintendent of Public Instruction: Tony Thurmond. The other front-runner is Marshall Tuck, who is a charter school advocate. I hate charter schools with the passion of a burning sun.

Local Measures

Regional Measure 3: YES. Even more hyper-local than the rest of this post, but mentioning it because it was a narrow call. Bridge tolls are regressive, and I'm not a big fan of raising them as opposed to, say, increasing property taxes (yes please) or income taxes. That said, taxing cars to pay for (largely) public transit is the direction the money should flow. It was thinly balanced for me, but the thrust of the projects won out over the distaste at the regressive tax.

2018-05-19: Free software log (April 2018)

This is rather late since I got distracted by various other things including, ironically, releasing a bunch of software. This is for April, so doesn't include the releases from this month.

The main release I worked on was remctl 3.14, which fixed a security bug introduced in 3.12 with the sudo configuration option. This has since been replaced by 3.15, which has more thorough maintainer testing infrastructure to hopefully prevent this from happening again.

I also did the final steps of the release process for INN 2.6.2, although as usual Julien ÉLIE did all of the hard work.

On the Debian side, I uploaded a new rssh package for the migration to GitLab (salsa.debian.org). I have more work to do on that front, but haven't yet had the time. I've been prioritizing some of my own packages over doing more general Debian work.

Finally, I looked at my Perl modules on CPANTS (the CPAN testing service) and made note of a few things I need to fix, plus filed a couple of bugs for display issues (one of which turned out to be my fault and fixed in Git). I also did a bit of research on the badges that people in the Rust community use in their documentation and started adding support to DocKnot, some of which made it into the subsequent release I did this month.

2018-05-14: Review: Thanks for the Feedback

Review: Thanks for the Feedback, by Douglas Stone & Sheila Heen

Publisher Penguin
Copyright 2014
Printing 2015
ISBN 1-101-61427-7
Format Kindle
Pages 322

Another book read for the work book club.

I was disappointed when this book was picked. I already read two excellent advice columns (Captain Awkward and Ask a Manager) and have read a lot on this general topic. Many workplace-oriented self-help books also seem to be full a style of pop psychology that irritates me rather than informs. But the point of a book club is that you read the book anyway, so I dove in. And was quite pleasantly surprised.

This book is about receiving feedback, not about giving feedback. There are tons of great books out there about how to give feedback, but, as the authors say in the introduction, almost no one giving you feedback is going to read any of them. It would be nice if we all got better at giving feedback, but it's not going to happen, and you can't control other people's feedback styles. You can control how you receive feedback, though, and there's quite a lot one can do on the receiving end. The footnoted subtitle summarizes the tone of the book: The Science and Art of Receiving Feedback Well (even when it is off base, unfair, poorly delivered, and, frankly, you're not in the mood).

The measure of a book like this for me is what I remember from it several weeks after reading it. Here, it was the separation of feedback into three distinct types: appreciation, coaching, and evaluation. Appreciation is gratitude and recognition for what one has accomplished, independent of any comparison against other people or an ideal for that person. Coaching is feedback aimed at improving one's performance. And evaluation, of course, is feedback that measures one against a standard, and usually comes with consequences (a raise, a positive review, a relationship break-up). We all need all three but different people need different mixes, sometimes quite dramatically so. And one of the major obstacles in the way of receiving feedback well is that they tend to come mixed or confused.

That framework makes it easier to see where one's reaction to feedback often goes off the rails. If you come into a conversation needing appreciation ("I've been working long hours to get this finished on time, and a little thanks would be nice"), but the other person is focused on an opportunity for coaching ("I can point out a few tricks and improvements that will let you not work as hard next time"), the resulting conversation rarely goes well. The person giving the coaching is baffled at the resistance to some simple advice on how to improve, and may even form a negative opinion of the other person's willingness to learn. And the person receiving the feedback comes away feeling unappreciated and used, and possibly fearful that their hard work is merely a sign of inadequate skills. There are numerous examples of similar mismatches.

I found this framing immediately useful, particularly in the confusion between coaching and evaluation. It's very easy to read any constructive advice as negative evaluation, particularly if one is already emotionally low. Having words to put to these types of feedback makes it easier to evaluate the situation intellectually rather than emotionally, and to explicitly ask for clarifying evaluation if coaching is raising those sorts of worries.

The other memorable concept I took away from this book is switchtracking. This is when the two people in a conversation are having separate arguments simultaneously, usually because each person has a different understanding of what the conversation is "really" about. Often this happens when the initial feedback sets off a trigger, particularly a relationship or identity trigger (other concepts from this book), in the person receiving it. The feedback giver may be trying to give constructive feedback on how to lay out a board presentation, but the receiver is hearing that they can't be trusted to talk to the board on their own. The receiver will tend to switch the conversation away to whether or not they can be trusted, quite likely confusing the initial feedback giver, or possibly even prompting another switchtrack into a third topic of whether they can receive criticism well.

Once you become aware of this tendency, you start to see it all over the place. It's sadly common. The advice in the book, which is accompanied with a lot of concrete examples, is to call this out explicitly, clearly separate and describe the topics, and then pick one to talk about first based on how urgent the topics are to both parties. Some of those conversations may still be difficult, but at least both parties are having the same conversation, rather than talking past each other.

Thanks for the Feedback fleshes out these ideas and a few others (such as individual emotional reaction patterns to criticism and triggers that interfere with one's ability to accept feedback) with a lot of specific scenarios. The examples are refreshingly short and to the point, avoiding a common trap of books like this to get bogged down into extended artificial dialogue. There's a bit of a work focus, since we get a lot of feedback at work, but there's nothing exclusively work-related about the advice here. Many of the examples are from personal relationships of other kinds. (I found an example of a father teaching his daughters to play baseball particularly memorable. One daughter takes this as coaching and the other as evaluation, resulting in drastically different reactions.) The authors combine matter-of-fact structured information with a gentle sense of humor and great pacing, making this surprisingly enjoyable to read.

I was feeling oversaturated with information on conversation styles and approaches and still came away from this book with some useful additional structure. If you're struggling with absorbing feedback or finding the right structure to use it constructively instead of getting angry, scared, or depressed, give this a try. It's much better than I had expected.

Rating: 7 out of 10

2018-05-13: Review: Twitter and Tear Gas

Review: Twitter and Tear Gas, by Zeynep Tufekci

Publisher Yale University Press
Copyright 2017
ISBN 0-300-21512-6
Format Kindle
Pages 312

Subtitled The Power and Fragility of Networked Protest, Twitter and Tear Gas is a close look at the effect of social media (particularly, but not exclusively, Twitter and Facebook) on protest movements around the world. Tufekci pays significant attention to the Tahrir Square protests in Egypt, the Gezi Park protests in Turkey, Occupy Wall Street and the Tea Party in the United States, Black Lives Matter also in the United States, and the Zapatista uprising in Mexico early in the Internet era, as well as more glancing attention to multiple other protest movements since the advent of the Internet. She avoids both extremes of dismissal of largely on-line movements and the hailing of social media as a new era of mass power, instead taking a detailed political and sociological look at how protest movements organized and fueled via social media differ in both strengths and weaknesses from the movements that came before.

This is the kind of book that could be dense and technical but isn't. Tufekci's approach is analytical but not dry or disengaged. She wants to know why some protests work and others fail, what the governance and communication mechanisms of protest movements say about their robustness and capabilities, and how social media has changed the tools and landscape used by protest movements. She's also been directly involved: she's visited the Zapatistas, grew up in Istanbul and is directly familiar with the politics of the Gezi Park protests, and includes in this book a memorable story of being caught in the Antalya airport in Turkey during the 2016 attempted coup. There are some drier and more technical chapters where she's laying the foundations of terminology and analysis, but they just add rigor to an engaging, thoughtful examination of what a protest is and why it works or doesn't work.

My favorite part of this book, by far, was the intellectual structure it gave me for understanding the effectiveness of a protest. That's something about which media coverage tends to be murky, at least in situations short of a full-blown revolutionary uprising (which are incredibly rare). The goal of a protest is to force a change, and clearly sometimes this works. (The US Civil Rights movement and the Indian independence movement are obvious examples. The Arab Spring is a more recent if more mixed example.) However, sometimes it doesn't; Tufekci's example is the protests against the Iraq War. Why?

A key concept of this book is that protests signal capacity, particularly in democracies. That can be capacity to shape a social narrative and spread a point of view, capacity to disrupt the regular operations of a system of authority, or capacity to force institutional change through the ballot box or other political process. Often, protests succeed to the degree that they signal capacity sufficient to scare those currently in power into compromising or acquiescing to the demands of the protest movement. Large numbers of people in the streets matter, but not usually as a show of force. Violent uprisings are rare and generally undesirable for everyone. Rather, they matter because they demand and hold media attention (allowing them to spread a point of view), can shut down normal business and force an institutional response, and because they represent people who can exert political power or be tapped by political rivals.

This highlights one of the key differences between protest in the modern age and protest in a pre-Internet age. The March on Washington at the height of the Civil Rights movement was an impressive demonstration of capacity largely because of the underlying organization required to pull off a large and successful protest in that era. Behind the scenes were impressive logistical and governance capabilities. The same organizational structure that created the March could register people to vote, hold politicians accountable, demand media attention, and take significant and effective economic action. And the government knew it.

One thing that social media does is make organizing large protests far easier. It allows self-organizing, with viral scale, which can create numerically large movements far easier than the dedicated organizational work required prior to the Internet. This makes protest movements more dynamic and more responsive to events, but it also calls into question how much sustained capacity the movement has. The government non-reaction to the anti-war protests in the run-up to the Iraq War was an arguably correct estimation of the signaled capacity: a bet that the anti-war sentiment would not turn into sustained institutional pressure because large-scale street protests no longer indicated the same underlying strength.

Signaling capacity is not, of course, the only purpose of protests. Tufekci also spends a good deal of time discussing the sense of empowerment that protests can create. There is a real sense in which protests are for the protesters, entirely apart from whether the protest itself forces changes to government policies. One of the strongest tools of institutional powers is to make each individual dissenter feel isolated and unimportant, to feel powerless. Meeting, particularly in person, with hundreds of other people who share the same views can break that illusion of isolation and give people the enthusiasm and sense of power to do something about their beliefs. This, however, only becomes successful if the protesters then take further actions, and successful movements have to provide some mechanism to guide and unify that action and retain that momentum.

Tufekci also provides a fascinating analysis of the evolution of government responses to mass protests. The first reaction was media blackouts and repression, often by violence. Although we still see some of that, particularly against out groups, it's a risky and ham-handed strategy that dramatically backfired for both the US Civil Rights movement (due to an independent press that became willing to publish pictures of the violence) and the Arab Spring (due to social media providing easy bypass of government censorship attempts). Governments do learn, however, and have become increasingly adept at taking advantage of the structural flaws of social media. Censorship doesn't work; there are too many ways to get a message out. But social media has very little natural defense against information glut, and the people who benefit from the status quo have caught on.

Flooding social media forums with government propaganda or even just random conspiratorial nonsense is startlingly effective. The same lack of institutional gatekeepers that destroys the effectiveness of central censorship also means there are few trusted ways to determine what is true and what is fake on social media. Governments and other institutional powers don't need to convince people of their point of view. All they need to do is create enough chaos and disinformation that people give up on the concept of objective truth, until they become too demoralized to try to weed through the nonsense and find verifiable and actionable information. Existing power structures by definition benefit from apathy, disengagement, delay, and confusion, since they continue to rule by default.

Tufekci's approach throughout is to look at social media as a change and a new tool, which is neither inherently good or bad but which significantly changes the landscape of political discourse. In her presentation (and she largely convinced me in this book), the social media companies, despite controlling the algorithms and platform, don't particularly understand or control the effects of their creation except in some very narrow and profit-focused ways. The battlegrounds of "fake news," political censorship, abuse, and terrorist content are murky swamps less out of deliberate intent and more because companies have built a platform they have no idea how to manage. They've largely supplanted more traditional political spheres and locally-run social media with huge international platforms, are now faced with policing the use of those platforms, and are way out of their depth.

One specific example vividly illustrates this and will stick with me. Facebook is now one of the centers of political conversation in Turkey, as it is in many parts of the world. Turkey has a long history of sharp political divisions, occasional coups, and a long-standing, simmering conflict between the Turkish government and the Kurds, a political and ethnic minority in southeastern Turkey. The Turkish government classifies various Kurdish groups as terrorist organizations. Those groups unsurprisingly disagree. The arguments over this inside Turkey are vast and multifaceted.

Facebook has gotten deeply involved in this conflict by providing a platform for political arguments, and is now in the position of having to enforce their terms of service against alleged terrorist content (or even simple abuse), in a language that Facebook engineers largely don't speak and in a political context that they largely know nothing about. They of course hire Turkish speakers to try to understand that content to process abuse reports. But, as Tufekci (a Turkish native) argues, a Turkish speaker who has the money, education, and family background to be working in an EU Facebook office in a location like Dublin is not randomly chosen from the spectrum of Turkish politics. They are more likely to have connections to or at least sympathies for the Turkish government or business elites than to be related to a family of poor and politically ostracized Kurds. It's therefore inevitable that bias will be seen in Facebook's abuse report handling, even if Facebook management intends to stay neutral.

For Turkey, you can substitute just about any other country about which US engineers tend to know little. (Speaking as a US native, that's a very long list.) You may even be able to substitute the US for Turkey in some situations, given that social media companies tend to outsource the bulk of the work to countries that can provide low-paid workers willing to do the awful job of wading through the worst of humanity and attempting to apply confusing and vague terms of service. Much of Facebook's content moderation is done in the Philippines, by people who may or may not understand the cultural nuances of US political fights (and, regardless, are rarely given enough time to do more than cursorily glance at each report).

Despite the length of this review, there are yet more topics in this book I haven't mentioned, such as movement governance. (As both an advocate for and critic of consensus-based decision-making, Tufekci's example of governance in Occupy Wall Street had me both fascinated and cringing.) This is excellent stuff, full of personal anecdotes and entertaining story-telling backed by thoughtful and structured analysis. If you have felt mystified by the role that protests play in modern politics, I highly recommend reading this.

Rating: 9 out of 10

2018-05-12: Review: Deep Work

Review: Deep Work, by Cal Newport

Publisher Grand Central
Copyright January 2016
ISBN 1-4555-8666-8
Format Kindle
Pages 287

If you follow popular psychology at all, you are probably aware of the ongoing debate over multitasking, social media, smartphones, and distraction. Usually, and unfortunately, this comes tainted by generational stereotyping: the kids these days who spend too much time with their phones and not enough time getting off their elders' lawns, thus explaining their inability to get steady, high-paying jobs in an economy designed to avoid steady, high-paying jobs. However, there is some real science under the endless anti-millennial think-pieces. Human brains are remarkably bad at multitasking, and it causes significant degredation of performance. Worse, that performance degredation goes unnoticed by the people affected, who continue to think they're performing tasks at their normal proficiency. This comes into harsh conflict with modern workplaces heavy on email and chat systems, and even harsher conflict with open plan offices.

Cal Newport is an associate professor of computer science at Georgetown University with a long-standing side profession of writing self-help books, initially focused on study habits. In this book, he argues that the ability to do deep work — focused, concentrated work that pushes the boundaries of what one understands and is capable of — is a valuable but diminishing skill. If one can develop both the habit and the capability for it (more on that in a moment), it can be extremely rewarding and a way of differentiating oneself from others in the same field.

Deep Work is divided into two halves. The first half is Newport's argument that deep work is something you should consider trying. The second, somewhat longer half is his techniques for getting into and sustaining the focus required.

In making his case for this approach, Newport puts a lot of effort into avoiding broader societal prescriptions, political stances, or even general recommendations and tries to keep his point narrow and focused: the ability to do deep, focused work is valuable and becoming rarer. If you develop that ability, you will have an edge. There's nothing exactly wrong with this, but much of it is obvious and he belabors it longer than he needed to. (That said, I'm probably more familiar with research on concentration and multitasking than some.)

That said, I did like his analysis of busyness as a proxy for productivity in many workplaces. The metrics and communication methods most commonly used in office jobs are great at measuring responsiveness and regular work on shallow tasks in the moment, and bad at measuring progress towards deeper, long-term goals, particularly ones requiring research or innovation. The latter is recognized and rewarded once it finally pays off, but often treated as a mysterious capability some people have and others don't. Meanwhile, the day-to-day working environment is set up to make it nearly impossible, in Newport's analysis, to develop and sustain the habits required to achieve those long-term goals. It's hard to read this passage and not be painfully aware of how much time one spends shallowly processing email, and how that's rewarded in the workplace even though it rarely leads to significant accomplishments.

The heart of this book is the second half, which is where Deep Work starts looking more like a traditional time management book. Newport lays out four large areas of focus to increase one's capacity for deep work: create space to work deeply on a regular basis, embrace boredom, quit social media, and cut shallow work out of your life. Inside those areas, he provides a rich array of techniques, some rather counter-intuitive, that have worked for him. This is in line with traditional time management guidance: focus on a few important things at a time, get better at saying no, put some effort into planning your day and reviewing that plan, and measure what you're trying to improve. But Newport has less of a focus on any specific system and more of a focus on what one should try to cut out of one's life as much as possible to create space for thinking deeply about problems.

Newport's guidance is constructed around the premise (which seems to have some grounding in psychological research) that focused, concentrated work is less a habit that one needs to maintain than a muscle that one needs to develop. His contention is that multitasking and interrupt-driven work isn't just a distraction that can be independently indulged or avoided each day, but instead degrades one's ability to concentrate over time. People who regularly jump between tasks lose the ability to not jump between tasks. If they want to shift to more focused work, they have to regain that ability with regular, mindful practice. So, when Newport says to embrace boredom, it's not just due to the value of quiet and unstructured moments. He argues that reaching for one's phone to scroll through social media in each moment of threatened boredom undermines one's ability to focus in other areas of life.

I'm not sure I'm as convinced as Newport is, but I've been watching my own behavior closely since I read this book and I think there's some truth here. I picked this book up because I've been feeling vaguely dissatisfied with my ability to apply concentrated attention to larger projects, and because I have a tendency to return to a comfort zone of unchallenging tasks that I already know how to do. Newport would connect that to a job with an open plan office, a very interrupt-driven communications culture, and my personal habits, outside of work hours, of multitasking between TV, on-line chat, and some project I'm working on.

I'm not particularly happy about that diagnosis. I don't like being bored, I greatly appreciate the ability to pull out my phone and occupy my mind while I'm waiting in line, and I have several very enjoyable hobbies that only take "half a brain," which I neither want to devote time to exclusively nor want to stop doing entirely. But it's hard to argue with the feeling that my brain skitters away from concentrating on one thing for extended periods of time, and it does feel like an underexercised muscle.

Some of Newport's approach seems clearly correct: block out time in your schedule for uninterrupted work, find places to work that minimize distractions, and batch things like email and work chat instead of letting yourself be constantly interrupted by them. I've already noticed how dramatically more productive I am when working from home than working in an open plan office, even though the office doesn't bother me in the moment. The problems with an open plan office are real, and the benefits seem largely imaginary. (Newport dismantles the myth of open office creativity and contrasts it with famously creative workplaces like MIT and Bell Labs that used a hub and spoke model, where people would encounter each other to exchange ideas and then retreat into quiet and isolated spaces to do actual work.) And Newport's critique of social media seemed on point to me: it's not that it offers no benefits, but it is carefully designed to attract time and attention entirely out of proportion to the benefits that it offers, because that's the business model of social media companies.

Like any time management book, some of his other advice is less convincing. He makes a strong enough argument for blocking out every hour of your day (and then revising the schedule repeatedly through the day as needed) that I want to try it again, but I've attempted that in the past and it didn't go well at all. I'm similarly dubious of my ability to think through a problem while walking, since most of the problems I work on rely on the ability to do research, take notes, or start writing code while I work through the problem. But Newport presents all of this as examples drawn from his personal habits, and cares less about presenting a system than about convincing the reader that it's both valuable and possible to carve out thinking space for oneself and improve one's capacity for sustained concentration.

This book is explicitly focused on people with office jobs who are rewarded for tackling somewhat open-ended problems and finding creative solutions. It may not resonate with people in other lines of work, particularly people whose jobs are the interrupts (customer service jobs, for example). But its target profile fits me and a lot of others in the tech industry. If you're in that group, I think you'll find this thought-provoking.

Recommended, particularly if you're feeling harried, have the itch to do something deeper or more interesting, and feel like you're being constantly pulled away by minutia.

You can get a sample of Newport's writing in his Study Habits blog, although be warned that some of the current moral panic about excessive smartphone and social media use creeps into his writing there. (He's currently working on a book on digital minimalism, so if you're allergic to people who have caught the minimalism bug, his blog will be more irritating than this book.) I appreciated him keeping the moral panic out of this book and instead focusing on more concrete and measurable benefits.

Rating: 8 out of 10

Last spun 2018-08-19 from thread modified 2008-08-13