Here's a very frightening thing that I've been meaning to do as a Halloween post. I realize that Halloween is for things that are fun scary, but it's not night yet, so this is isn't really Halloween.
You may have heard of Goodhart's Law. It has nothing to do with being good-hearted, Goodhart's just the name of its author.
Here's my current formulation: Any measurement which is used to guide policy will become corrupt.
You understand? Not might become corrupt. Not won't become corrupt if it's well designed. WILL BECOME CORRUPT.
Are you shaking? You are living in a culture which gives more and more trust to using measurement to guide policy.
Good long discussion of metrics (metrics are measurements used to guide policy)
The Wells Fargo accounts scandal is a classic example of measuring the worng thing-- top management set up impossible demands for new accounts, and staff both pressured customers into getting accounts they didn't need and also added accounts to customers who didn't ask for them. This has turned into a disaster for Wells Fargo.
This sort of thing doesn't just happen in business. If there's a demand for crime to go down, it might just get translated into make crime statistics go down, so the public is discouraged from reporting crime.
Does GDP measure somehting important? Do unemployment statistics?
I think you can get a rough measure of which places are better to live in than others by looking at immigration/emigration and adjusting it for the level of risk and cost people are willing to endure in order to move.... but if this were used for policy it would go wrong some way or other.
This entry was posted at http://nancylebov.dreamwidth.org/1085632.html. Comments are welcome here or there. comments so far on that entry.