The best chess player in the world is a grandmaster with a computer. Not Deep Blue, or Kasparov, or Anand. It turns out that it’s the same way with predicting elections: smart people using good models. As you have no doubt heard, a number of computer-based models correctly picked the winner in all (or nearly all) fifty states +DC in the 2012 US presidential election. Analytics FTW. This is nothing new (see 2008), but this success received increased attention this fall due to controversy in the weeks leading up to election day.
In particular, Nate Silver, creator of one of the most well-known prediction models and author of fivethirtyeight.com, was singled out by several media figures including Joe Scarborough, Karl Rove, and Dylan Byers. Silver uses state and national polls and other factors to create a statistical model that predicts state-level vote totals (and therefore projected electoral vote counts). The most basic objection to Silver’s methods was that his predictions were unrealistic and implausibly precise. Scarborough:
Nate Silver says this is a 73.6 percent chance that the president is going to win? Nobody in that campaign thinks they have a 73 percent chance — they think they have a 50.1 percent chance of winning. And you talk to the Romney people, it’s the same thing," Scarborough said. "Both sides understand that it is close, and it could go either way. And anybody that thinks that this race is anything but a tossup right now is such an ideologue, they should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they’re jokes.
You’ve got to be careful about these polls," Rove said. "We endow them with a false scientific precision they simply don’t have.
Dylan Byers, who writes about political media for politico.com, characterized Silver’s work as “little more than poll aggregation”, which I find akin to characterizing Warren Buffett’s work as little more than reading quarterly reports.
Author Geoffrey Dunn claimed that Silver does not understand the underlying dynamics underlying presidential politics:
Silver’s commitment to a quantitative, value-free approach to the living, breathing universe–not only in politics, but in sports, entertainment, even dating–with an emphasis on numbers can be troubling, to the point of absurdity, when the answers have nothing to do with statistical equations…Here’s a guy who’s never been centrally involved in a national election, whose political acumen comes from a calculator, and, who, I am willing to bet (and I haven’t seen the numbers) has never organized a precinct in his life, much less walked one, pontificating about the dynamics in the electoral processes as if he actually understood them.
The question of how human values should be reflected in quantitative models is fascinating and relevant, but sadly Dunn chooses to lecture rather than engage.
It so happens that this time out, most of the initial criticism came from Republicans. My point is not to single out Republicans but rather those who are so dismissive of the value of analytical model for predicting future behavior. They’re all over the map politically.
The underlying tension is only superficially about numbers. It’s fear of an outsider who claims to possess wisdom about presidential politics whose justification is based on concepts beyond the reach of the current establishment. Silver is a threat to pundits like Scarborough and Rove and writers like Byers and Dunn, and his success needs to be discredited because it threatens to make them obsolete. What good are pundits if they are outside of the process, do not report on the principals involved, and have no special insight regarding what will happen? As a result, Silver is treated the same way as many politicians of both parties: built up under false premises so that they can later be torn down.
In the aftermath of the (attempted) Silver takedown, defenders fired back and another set of Silver critics emerged. These critics are more ideologically diverse, ranging from libertarian Colby Cosh to conservative Jonah Goldberg to liberal Alexis Madrigal at the Atlantic. The objections are frustratingly simple minded. The common thread is that the critic takes issue with Silver’s defenders rather than making an actual critique on his methods:
The left’s celebrating the analytical method right now as if it belonged to them. But it doesn’t.
The situation is that many of Nate Silver’s attackers don’t really know what the hell they are talking about. Unfortunately, this gives them something in common with many of Nate Silver’s defenders, who greet any objection to his standing or methods with cries of ‘Are you against SCIENCE? Are you against MAAATH?’ If science and math are things you do appreciate and favour, I would ask you to resist the temptation to embody them in some particular person.
To listen to many of Silver’s defenders, questioning his methodology is akin to rejecting evolution or the laws of thermodynamics, as if only his model is sanctified by the god Reason.
The same dynamic emerged inside the Oakland A’s, and baseball generally, according to Moneyball. (I mentioned Moneyball in an analytics post! What do I win?)
The feeling I am left with is annoyance mixed with disinterest. (Sorry.) Annoyance because it is frustrating to witness interesting conversations very nearly happening, and disinterest because I feel that those who act innumerate are allowed to have an opinion, but there is no good reason for me to listen to them. Silver himself often seems to take this attitude: get on with the work instead of talking about it. In the long run, I hope this “ignore the haters” attitude is disruptive because it builds a body of evidence that begins to become irrefutable, as has happened with sports analytics. There’s no point in asking journalists to learn stats, unless you are doing the hiring. This is the flaw of the academic: if only people just thought like I do – how do I make them think like me?
A more flattering but still counterproductive attitude is to view quants as mystics. Cubicles become secret worlds, computer screens become portals, and models become McGuffins. This characterization of analytics in the Obama campaign is typical: well-written, engrossing, and a mixed blessing if you wish to see the role of analytics in society advanced. Ultimately I’d like the view to be: well, yeah, that’s kinda how you do it.
Sometimes people expect an analysis to be backed by mystical, unknowable methodology. If it is too complicated for me to understand, it must be right. In other cases subtleties of approach are lost in translation. Silver’s methodology page lays out an ornate set of rules, but based on a simple principle: the primary inputs of the forecast are polls. It’s far easier to skim and say “he’s doing poll aggregation”. On the other hand a substantive objection to Silver’s approach is that simpler methods have had the same success using measures that count. It’s too soon to say, so the answer is to continue to dig deeper rather than finger point or simply exclaim “MAGIC!”
The victories won by analytics are hard fought and often ignored, perhaps because the marketplace demands adaptation or death. The reward for a job well done is survival and perhaps a promotion. Weather prediction has secretly gotten much, much better over the past two decades. Marketing has been completely transformed. Same with logistics, and the list goes on and on. A prerequisite for success is that those who understand and practice analytics have a seat at the decision making table, but this may look different from what we in the analytics community envision. In the long run, many of us will change our identities and build new muscles. No longer will it be “management”, “IT” and the “math geeks”. Like LeBron James who can run past you, jump over you, and barrel you over, the leaders of the future will be MBAs who use “confidence interval” without air quotes, hackers with slide decks, and R coders with good hygiene and social graces. In short: chessboxers.