2007-08-01: mdfrm-utf8 1.1
Tonight I got my first spam that used RFC 2047 encoding in a character set
that Encode couldn't recognize. In retrospect, I'm surprised it took this
long. It turns out that Encode throws an exception when this happens, so
mdfrm-utf8 now catches that exception and displays the raw encoded header
as the best available option.
You can get the latest version from the
mdfrm distribution
page.
2007-08-02: Long time no write
So, as you may have noticed, I haven't been posting much lately, which is
because I've been pretty much swamped.
Mostly, it's work. We've had three people resign from the group I work
with at Stanford over the past four months or so, which has left us
severely short-staffed given that we were down two bodies before that. I'm
also trying hard to finish up a major component of the largest work project
I've ever tackled (completing our migration to Kerberos v5 and turning off
Kerberos v4), and after lots of testing false starts and lots of tweaks,
we're finally on target to switch over our administrative master and the
source of all password propagation to other environments to Kerberos v5.
As you can imagine, I have a list of things to be finished and tested a
mile long.
Then, last week, I finally got a much-needed vacation, during which I was
planning on doing lots of things for non-work projects like Debian and
during which I actually did almost nothing beyond talk to my friends,
write, and read. Which was the right choice, as it always is when that
happens, but which doesn't help any in getting caught up.
So, I have a ton of OpenAFS patches to apply, a new release of remctl to
get out, a new release of kstart to get out, Debian Policy work that
desperately needs attention, a new release of lintian that needs to go out,
various non-Kerberos bits of work, five book reviews to write for things I
finished on vacation, two book reviews and a magazine review already
written that need to be posted, and doubtless other things that I'm
forgetting. At least I now have most of that written down so that I can
start prioritizing.
If all goes according to plan, that means that the activity increase will
turn into a temporary flurry as I get caught up with things. I'm hoping
for a very productive month. Of course, I also just got a new digital
camera, and another book order just shipped.
Really, about four weekends in a row would be lovely right now. Not
vacation, since that's a different-feeling sort of thing with time
stretching out in front of one and an obligation to use those precious
vacation days to really relax. No, I want the sort of short, quality
non-work time that I get on weekends, where I get various bits of non-work
work done and catch up on to-do lists. I just want four of them in a row.
2007-08-04: New camera
I finally took the plunge and got a Canon PowerShot SD750, and finally this
weekend had a chance to start playing with it. Here are a few notes on
using it with Debian.
Gnome (gnome-volume-manager, in particular) just deals with the camera
quite well. Connect it to its USB cable, connect the USB cable to the
computer, and gnome-volume-manager pops up a dialog box asking if you want
to import the pictures. You do need to have gthumb installed or importing
does nothing without an error message.
The default high-quality large size is probably overkill. I'm not sure if
I'm going to keep the huge images or not. ImageMagick is, as usual, my
favorite program to do resizing and similar transformations (specifically
the convert command-line utility). The camera writes out the standard Exif
image attributes. After looking through the available options for an Exif
tag editor, I picked exiv2 since it would let me edit Exif comments.
Unfortunately, you have to know the tag names in order to use it (some
shortcuts for common options would be nice), but there are useful examples
in the man page.
The first application I wanted it for was to get cover images for book
reviews when the publisher and similar information databases don't already
conveniently have one. That took a bit of experimentation, since I can't
ues flash (it reflects too much off the cover) and it's hard to get enough
light otherwise for the camera to not have to increase the shutter speed
and hence blur the picture if you're holding the camera in your hand.
Putting the books on a TV tray on my balcony in the direct sunlight worked
out wonderfully, though, and I'm very happy with the results. I resized
the pictures to 1024x768, then used ImageMagick's display program to crop
them and shrunk them down to the size I use for thumbnails on review pages.
For the most part, except where I didn't get the book quite square, you
can't tell the difference between my pictures and cover art from the
publisher.
I'm probably not going to use the camera very much regularly, as I'm not
that much of a camera person, but being able to capture cover images is
really cool. And it can go with me the next time I go on vacation. I
might try taking some video of waves crashing over rocks at Yachats during
my October vacation.
2007-08-07: krb5-sync 0.7
It turns out that if you pass an unqualified principal without an instance
into the kname_parse function of MIT Kerberos's libkrb4 compatibility
library, it just leaves the instance set to whatever junk was originally in
the buffer. That was the reason why propagation to our AFS kaserver
environment wasn't working in pre-prod, because I started using our
production AFS kaserver administrator instead of a custom one I'd created
for testing.
I also rediffed a clean patch for MIT Kerberos 1.4.4 and fixed a few other
minor issues. This code is going into production on Saturday.
You can get the latest version from the
krb5-sync
distribution page.
2007-08-08: kadmin-remctl 1.8
This is the first announced release of kadmin-remctl, although it's been
available from my archives site for a while. This is a remctl front-end to
the Kerberos kadmin functions to create, delete, enable, and disable
accounts, change passwords, and so forth. We export this via a remctl
interface for the use of other middleware at Stanford that can't cope with
the direct kadmin protocol easily. We also provide a tool that will (after
Saturday's upgrade) be based on remctl that the Help Desk uses to reset
passwords for users, with various security checks.
This is the code that we're going into production with on Saturday and it
works reasonably well, although it still leaves something to be desired in
cleanliness and structure. For example, it uses the command-line OpenLDAP
clients to modify Active Directory rather than using a native Perl LDAP
implementation.
Hopefully, I'll have time to clean it up and improve it in the future.
There's also a bit more work that needs to be done to support instance
creation to work around iPass's completely broken security model.
You can get the latest version from the
kadmin-remctl
distribution page.
2007-08-09: Latest haul
There was a new Elizabeth Bear book out, so it was time for another
Powell's order.
Elizbeth Bear -- Undertow (sff)
Ellen Kushner & Delia Sherman -- The Fall of the Kings
(sff)
Ken MacLeod -- The Execution Channel (sff)
Sarah Monette -- The Virtu (sff)
Edward Tufte -- Beautiful Evidence (non-fiction)
Joss Whedon, et al. -- Serenity: Those Left Behind (graphic
novel)
I also got a copy of Kushiel's Dart in hardcover, and I just ordered
the second and third books in hardcover. I decided I really wanted to own
the whole series in hardcover rather than paperback. I may well re-read
them sometime soon, maybe after the second trilogy concludes.
2007-08-11: Kerberos upgrade
It's finally done, which means that the hardest part of the Kerberos
project is done. Today, we switched our master realm from Kerberos v4 to
Kerberos v5, which meant putting
krb5-strength
and krb5-sync into
production, replacing our Kerberos v4 kadmind with a server that still
handles examine, enable, and disable but proxies password changes to
Kerberos v5, and replacing our kadmin remctl interface with
kadmin-remctl.
I also replaced the Help Desk password change client.
It all seems to have worked. Some of the systems (particularly the K4
authentication servers) are in a slightly weird state now, and lots of
updated documentation needs to be written, but all the coordination,
testing, and 27-step rollout is finished.
It's quite a relief. I'm now going to do as little as possible work-wise
for the rest of the weekend, and then figure out what's next on the agenda.
2007-08-12: Glory Road
Review: Glory Road, by Robert A. Heinlein
Publisher: Avon
Copyright: 1963
Pages: 288
Pfc. Gordon, the first-person narrator who later picks the name Oscar, is a
Vietnam vet given his discharge after being wounded. He heads for Europe,
deciding to try for a degree from a European university paid for by some
lottery tickets he won in a poker game. But while living cheaply (and
mostly nude, Heinlein's regular obsessions showing in the background) on an
island, waiting for lottery drawings, he meets a rather remarkable woman.
And after discovering his lottery ticket is a fake, he answers an ad in the
paper that seems targetted at him, meets her again, and ends up in a very
different world.
The first part of this book is somewhat enjoyable. It doesn't have much of
a plot, and it's full of Heinlein's typical off-hand political and social
commentary, but it has a breezy, conversational tone that's quite readable.
It helps that Oscar is self-deprecating and has a good sense of humor
about his life.
Unfortunately, this only lasts until Oscar is taken into another dimension
by the horribly attractive female, who he calls Star. He is hired to be a
hero, for a great reward at the end of a dangerous mission. He gets led by
the nose into various dangers, which he overcomes. Star manages to stay
vague about exactly what that mission is, while bantering with Oscar in
that annoying Heinlein fashion that reads like some combination of slang,
excessive chivalry, jaw-dropping sexism, weird politeness, and exaggerated
teasing.
And that's most of the book.
Oh, things happen. Oscar finds ways of defeating various monsters through
quick thinking and ingenuity. He runs afoul of local customs by being
unwilling to sleep with naked women who are parading themselves in front of
him, in a pointed commentary on how dumb it is that Earth customs do not
match Heinlein's sexual fantasies. And there is a great deal of annoying
love story as Oscar falls for Star (he can't help it; she's declared to be
irresistable by the author), subjects her to constant exaggerated
compliments, threatens to spank her and treat her like a child, puts her
firmly in her place a few times, and otherwise behaves like the typical
sexist Heinlein hero. All of which is apparently horribly charming in her
culture; either that, or she's so smitten with authorial fiat that she
can't figure out he's acting like an abusive twit. As is sadly typical for
these sorts of things, any real mutual understanding apparently pales in
importance next to being the sort of hero and heroine who fall in love with
each other.
But all of that is the standard sort of thing that happens around the plot,
and there's basically no plot. They're on a quest to do something, about
which Star is excessively coy but which eventually turns out to be stealing
the Foozle from the Guarded Tower. (I think it was called something other
than Foozle, but it was hard to care.) To the shock of absolutely no one,
Oscar succeeds, with sufficient help from Star and her weird servant that
one has to wonder what they needed him for. Indeed, it's so painfully
obvious that Oscar is the least useful member of this party that even Oscar
has to wonder what they needed him for. Upon asking that, he's given a
remarkably silly response that doesn't actually answer the question.
After they find the Foozle, the book takes a sharp turn towards interesting
for the last eighty pages or so. We finally find out what the hell is
behind this adventure and why Star cares about the Foozle (although by
"find out" I mean "are smacked upside the head with the Wet Trout of
Exposition"). We get a bit more of Heinlein utopian government, which
while remarkably impractical does manage to be sarcastic commentary on the
governments we have. And we finally get a bit of character development as
Oscar discovers some problems with life as an ex-hero. Unfortunately, that
character development is rooted heavily in the relationship between Oscar
and Star, and realistic relationship turmoil is not one of Heinlein's
strong points; it's a battle between interesting and painful and painful
mostly wins. But to give him credit, Heinlein does go for a somewhat
non-traditional denouement and clumsily pokes at the happily-ever-after
ending to see if it holds up.
The beginning and the ending both have the seeds of a good book. But,
alas, the middle of Glory Road is page after page of this sort of
thing:
"Oscar, by your standards — the way you have been raised — I am
a bitch."
"Oh, never! A princess."
"A bitch. But I am not of your country and I was reared by another code.
By my standards, and they seem good to me, I am a moral woman. Now... am I
still 'your darling?'"
"My darling!"
"My darling Hero. My champion. Lean close and kiss me. If we die, I
would my mouth be warm with your lips. The entrance is just around this
bend."
"I know."
A few moments later we rode, swords sheathed and bows unstrung, proudly
into the target area.
Best used for a drinking game in double entendre; within pages, you'll be
blissfully unaware of just how bad the writing is. If you want to read 288
pages of that, by all means, have at it. I have a copy you can have.
Rating: 3 out of 10
Permanent
review page
2007-08-13: krb5-sync 1.0
Last Saturday, I deployed all this code in production at Stanford, so it's
no longer beta software. This releases cleans up some logging issues and
filters out common errors so that I'm not deluged with e-mail from accounts
created in Kerberos and not in Active Directory because they haven't yet
been sponsored. It also fixes a few other minor logging issues.
You can get the latest version from the
krb5-sync
distribution page.
2007-08-17: Investing and betting
A recent article in Salon repeated the annoying comparison between
investing and betting as a way of explaining the current mortgage collapse,
which annoyed me enough to send a letter, and I'm apparently not done
talking about this. Warning: this will be long.
I think calling investment betting is fundamentally broken. Worse than
that, I think it betrays a fundamental lack of understanding of economic
principles, and while that's to be expected from a lot of individuals,
having a columnist whose job it is to explain subtle issues to people use
the same broken analogy is disappointing and decreases the economic
understanding of the readers.
You can stretch the definition of betting to mean any time you do something
that may or may not make more money. Working hard to finish a project so
that you get a bonus is in some sense "betting" that you'll finish the
project and the work will be worth it, and sometimes people call it that.
In that sense, investment (and many other things) is betting. But when
someone calls investment betting, they don't just mean that. They imply
it's like sports betting: it's speculation on some fundamentally
meaningless activity, it's risky, winning or losing a bet doesn't do
anything useful for the world, and it's vaguely shady. And, importantly,
betting is something you can not do, and that indeed most people don't and
shouldn't do. It's something people do for entertainment with money they
can lose.
That last is what makes this analogy so broken. The implication is that
investing (or at least some form of investment that the author doesn't
like) is something that we can not do. And that's simply wrong. Our
economy, any modern economy, would not survive without investment. Many of
these investments that people complain about are natural consequences of
markets; it would be almost impossible to prevent people from making these
investments without draconian legislation, and passing such legislation
would cripple things that free markets are quite good at.
Let's talk about some basics. Broadly speaking, you can divide investments
into two basic types. Either you purchase something, or a share of
something, that you think will retain its value and possibly grow in value
(real estate, gold, stock in a company); or you loan money to someone who
will pay you interest in exchange for the loan (bonds, treasury
certificates, bank deposits).
Everything you can do with money that you're not spending, other than
putting it all in your mattress, is investment. If investment is gambling,
almost everyone in the world is gambling. Even depositing your money in
the bank is investing. It's a loan to the bank, an investment of the
second type. The bank pays you interest in exchange for the right to use
your money (generally loaning it out to someone else at a higher rate of
interest). It's a loan that, for most accounts, the bank promises to repay
on demand whenever you want, but that's still a loan, just a fairly safe
one.
This is where risk comes in. Risk is easiest to think about with loans.
It is, simply, the chance that the person or entity to whom you loaned
money will actually pay it back. We're all used to this: when you loan
money to a friend, there's always a chance that they won't pay you back.
You take that into account when you decide to make the loan. Just because
there's a chance they won't pay you back, that doesn't mean that you're
gambling; you're taking a "calculated" risk, which means that you're
evaluating the level of risk and deciding that it's worth it to you.
Risk is a bit harder to think about when it comes to buying property, but
think of buying a house. You're running various risks when you do that.
The house may have some undisclosed structural problem that you'll then
have to pay a lot of money to fix. Someone might build an airport next to
the house, making the area very noisy and reducing the value of the house.
Anything that you buy has this problem. Most things we buy to use and
expect to throw them away when we're done. Houses are the main thing
(outside of pure investments like gold or stocks) where we're used to
thinking about both the use of the house and the value when we sell it
later.
Obviously, risk is bad. Risk means you could lose something: the money of
the loan, which is never paid back, or the value of the house, which was
lost when someone built a trash dump next to it. The basic idea of an
investment market is that when you take a risk, you should be compensated
for it with higher profit if the risk pays out. This is for obvious
reasons. Risk is bad, so unless people make more money from higher risk,
they simply won't do things that are high-risk at all. If everyone always
took the minimum possible risk, only people who had absolutely perfect
credit with million-dollar incomes would be able to buy a house. We want
people to take higher-risk activities occasionally; it's good for the
economy, it lets average people buy houses, and it lets companies try
things that may or may not work. So, in exchange, people are paid more for
taking a higher risk, to give them some incentive for doing so. Someone
who isn't a millionaire (more likely to not pay back a loan, and therefore
is higher risk) pays higher interest to compensate the person loaning them
money for taking a higher chance they won't be paid back.
Now, some people need all of their money and don't want to take any risk.
They can deposit their money in federally-insured accounts, taking as low
of risk as we can offer for an investment. In return, though, they get
minimum income from that investment. People who take higher risk (buying
stock, for instance, which runs the risk that a company may cease to be
profitable and its assets won't be worth what the company is valued for on
the market) get higher income.
And this is where we get to the part that people have the most trouble
with, I think. Investments aren't just made and held all the time.
They're bought and sold constantly, which can be confusing. But one of the
major reasons for this is the notion of valuation. The basic economic
principle at work is that you never really know how much something is worth
until you sell it. Think of some family heirloom — you can have it
appraised, you can guess, but you never know how much it's truly worth
until you find someone who wants to buy it.
In order to make useful decisions about which companies are worth owning,
or which loans are paying enough to be worth their risk levels, someone has
to evaluate those companies or those loans and communicate how valuable
they are. And you never know the results of that investigation until that
person buys or sells for a particular price. That price reflects their
belief of how much something is worth. Prices are communication of other
people's knowledge.
An example. If you have a loan that, when held, returns 7%, and no one
wants to buy it out for the price that you're offering, that means the
market has decided that the risk level is high enough that 7% isn't enough.
If you want to sell it, you'll have to sell it for less than the original
value of the loan. That means that, for the new buyer, the loan will pay
8% of what they paid for the loan instead of 7%. See what happened? The
process of selling the loan has adjusted the rate of return to match what
the market decided the risk was. It fixed something that was inaccurate,
and afterwards we have better information.
Markets are exceptionally good at this. In the long run, markets arrive at
excellent estimates of exactly how much something is worth, or of exactly
how much risk there is in a given loan. They do this through sheer
self-interest on the part of the investers: if you can evaluate the worth
of something better than other people, you buy things that are too cheap
and sell things that are too expensive, and you make lots of money. And in
the process of making all that money, you end up adjusting the market
prices to be more accurate. This is the purpose of investment markets in a
nutshell.
This is valuable and important work! If it weren't for investment markets,
we would have no idea how much someone should have to pay for a loan given
their chances of paying it back, or whether a company's business model is
good enough to be worth investing in. The market enables millions of
people to make decisions about their areas of expertise, collects all that
information, and reflects it in prices with a high level of accuracy.
Sounds less and less like gambling, doesn't it? It is a little like
gambling in that gambling odds serve the same purpose, and similarly arrive
at very accurate estimates of probabilities. But unlike the gambling that
people think of (sports, poker, blackjack), investments place a value on
something that's real, and having that valuation helps the economy work
better because the economy now has more information.
The normal, competent invester is doing something that looks nothing like
gambling. They're buying investments to hold for the long term, based on
their desired level of risk and profit. If they have a high risk
tolerance, they buy things that have a high level of risk and pay a
correspondingly high rate of return. If they have a low risk tolerance,
they buy things like US treasury bonds that have a low rate of return but
very little risk. It's really that straightforward.
Whenever you see people freaking out about investment markets or claiming
that some investment is gambling, remember that the purpose of an
investment market is to value something by buying and selling it and figure
out what's being valued. For example, the recent market chaos is over home
mortgages, specifically "sub-prime" mortgages (loans to people with bad
credit). The securities that are being traded are essentially pieces of
home mortgages, pieces of loans. We're discovering that many buyers in the
past thought that these loans were lower risk than they actually are
(probably because we've had an economic boom and an excess of borrowing
that's now ending). Now, the market is doing what markets do and doing a
mass revaluation of all those loans. Previous buyers are discovering that
they were wrong about the value of the loans, or in other words that they
weren't being paid enough interest on their investment to have the risk be
worth it. New buyers are purchasing the loans at more accurate values (and
hence higher interest rates for them). This shakeout is exactly what
markets are for. At the end, we'll end up with a much more accurate view
on what those mortgages are really worth.
Figure out what the market is putting a value on and suddenly the
mechanisms make sense.
One final thing. There are certain types of investments that get an even
worse bad rap, that people claim are just gambling. An example would be
Dow Jones futures, where people buy and sell securities that represent the
chances that the Dow Jones average will go up or down at the start of the
next trading day. Isn't that gambling? Aren't people just making bets on
their guesses of what the Dow will do, with no purpose outside of that
betting?
No. Look at what this market is valuing. The Dow is a collection of stock
in a bunch of companies (that happen to be considered the core companies of
the economy in some sense). Stock in those companies only trades for a
short period of time during each day. However, economic activity and news
about companies doesn't stop when the market closes. What the futures
market does is let people continue that valuation process while the market
is closed, which provides useful information to the people in the market at
the start of the next day and feeds in to the overall process of valuing
all those companies and the economy as a whole. This kind of activity will
happen. It can't not happen. People don't stop caring about the value of
things while the stock market is closed. They will create ways to act on
those opinions and additional information whenever they want, and the
futures market in stock market indexes is part of that.
2007-08-25: Obvious security
It occurs to me, after a ton of fake YouTube links that appear to be the
spam du jour, that quite a lot of phishing would be stopped completely by
the simple expedient of all HTML-enabled mail clients looking at <a>
tags. If the text of the anchor looks like a URL and that URL doesn't
match the actual target URL, throw all sorts of bright red warnings all
over the page, remove the link, and so forth.
I can't think of any drawbacks to this and a good 80% of phishing messages
I see would be caught by that. And yet, I don't remember hearing about
clients doing this. Why not?
2007-08-26: remctl 2.10
I finally found time this weekend to incorporate a bunch of additional work
by Marcus Watts, including a Java client that speaks protocol v2 (yay!) and
a Java server. I also fixed a ton of minor bugs that I'd been saying "next
release" for. I really went rather too long between releases this time, so
it's good to get a release out.
Expect more in the not too distant future, since several other schools seem
to be deploying remctl and feeding me patches. Very gratifying, that.
You can get the latest version from the
remctl distribution
page.
2007-08-27: krb5-sync 1.1
MIT Kerberos uses an annoying structure for Kerberos principals. Rather
than giving you nul-terminated strings, all the portions of a Kerberos
principal are krb5_data structs with a length attribute. It is nice in
that it means principals could contain nul characters if anyone wished, but
it's hard to deal with in C.
It turns out that kadmin nul-terminates principal instances, and the
previous krb5-sync versions were relying on that. However, when processing
kpasswd protocol requests, it doesn't, which was causing root instances to
not be propagated properly into Kerberos v4 at Stanford (and may be behind
some of the crashes we were seeing in kadmind). My previous code for
checking the instance for propagation also didn't cope with some cases
where the instance was a substring of an allowed instance.
I rewrote the whole checking loop as a state machine to fix both of these
problems, and now it seems to work.
You can get the latest version from the
krb5-sync
distribution page.
All posts