Bruce Bueno de Mesquita is a political scientist specializing in game theory and coalitions. Unlike most academics, he's made a business out it. He routinely consults for the government and businesses trying to predict future political events. He's got a new book out -- The Predictioneer's Game -- which I suspect may be the next Freakonomics. Controversial, widely read, simultaneously powerful and a little gimmicky.
I've discovered that BDM is a polarizing figure among game theorists. On the one hand, he's clearly good at what he does, and he's succeeded in bringing some popular attention to it. On the other hand, he sometimes comes off as arrogant, and he makes some really big claims. Lots of people think that one of these days, he's going to be wrong about something big. I get the impression that a jealous few would like to see it happen.
A bunch of fun links to BDM-related material:
* His interview on the Daily Show on Monday of this week. In my opinion, Stewart asks good questions, but lets him off easy on the answers.
* The TED talk where he first (?) made his prediction that Iran will not get the bomb.
* A really good NYTimes article on the topic.
There's one link I couldn't find: video from the History Channel's show "The Next Nostrademus." I'm told it's over the top. Links anyone?
Wednesday, September 30, 2009
Tuesday, September 29, 2009
Follow up: Big stories from the '08 election
About a month ago, I posted asking for your thoughts on the biggest stories of the 2008 presidential election. If you recall, my goal was to compare your intuition with a computer algorithm chewing through reams of newspaper archives.
Here are the results, as a line graph. The red line is an aggregate tally of your responses. The blue line tracks total words of coverage devoted to the campaign. The two track reasonably well, taking into account the fact that news coverage often lagged actual events by a day. (I used newspaper coverage for this tally; presumably, evening TV coverage wouldn't have had this lag.)
When you chalk up the important stories, here's what you get. These are the top 10 stories from the election, tallied using three slightly different measures.
Your Responses
1. Reverend Wright controversy
2. Rep. convention; Palin joins ticket
3. Election day
4. Super Tuesday
5. Clinton concedes Dem. nomination to Obama
6. Iowa caucus
7. McCain suspends his campaign to work on the financial crisis
8. Obama makes his “PA bitter” comment
9. Dems vote to seat MI and FL delegates
10. Dem. convention
Computer version A (text volume)
1. Election day
2. Super Tuesday
3. Election week
4. Iowa caucus
5. NV and SC caucuses
6. Obama becomes presumptive Dem. nominee
7. McCain becomes presumptive Rep. nominee
8. Rep. convention; Palin joins ticket
9. Giuliani and Edwards withdraw
10. Dem. convention
Computer version B (IEM trading volume)
1. Election day
2. Rep. convention; Palin joins ticket
3. Final phase of the election
4. Election week
5. WA, D.C., MD, VA primaries
6. McCain becomes presumptive Rep. nominee
7. Super Tuesday
8. Democratic convention
9. NC and IN primaries
10. OR and KY primaries
What you see is that the computer gets a lot of things right (election day, Super Tuesday, the party conventions), but misses nuances that a lot of people thought were important. For example, the Rev. Wright controversy was the number one story in your responses -- almost everyone remembered it as a pivotal event in the campaign -- but it didn't generate that much media coverage, so the computer didn't see it as particularly important. I think it's an open question whether our memories are cloudy, or the computer's event detection is flawed.
If you're really into this stuff, here's a link to the paper. It's half baked, but feedback from the conference convinced me that the concept is worth pursuing as a research topic. Thanks again for your help!
Here are the results, as a line graph. The red line is an aggregate tally of your responses. The blue line tracks total words of coverage devoted to the campaign. The two track reasonably well, taking into account the fact that news coverage often lagged actual events by a day. (I used newspaper coverage for this tally; presumably, evening TV coverage wouldn't have had this lag.)
When you chalk up the important stories, here's what you get. These are the top 10 stories from the election, tallied using three slightly different measures.
Your Responses
1. Reverend Wright controversy
2. Rep. convention; Palin joins ticket
3. Election day
4. Super Tuesday
5. Clinton concedes Dem. nomination to Obama
6. Iowa caucus
7. McCain suspends his campaign to work on the financial crisis
8. Obama makes his “PA bitter” comment
9. Dems vote to seat MI and FL delegates
10. Dem. convention
Computer version A (text volume)
1. Election day
2. Super Tuesday
3. Election week
4. Iowa caucus
5. NV and SC caucuses
6. Obama becomes presumptive Dem. nominee
7. McCain becomes presumptive Rep. nominee
8. Rep. convention; Palin joins ticket
9. Giuliani and Edwards withdraw
10. Dem. convention
Computer version B (IEM trading volume)
1. Election day
2. Rep. convention; Palin joins ticket
3. Final phase of the election
4. Election week
5. WA, D.C., MD, VA primaries
6. McCain becomes presumptive Rep. nominee
7. Super Tuesday
8. Democratic convention
9. NC and IN primaries
10. OR and KY primaries
What you see is that the computer gets a lot of things right (election day, Super Tuesday, the party conventions), but misses nuances that a lot of people thought were important. For example, the Rev. Wright controversy was the number one story in your responses -- almost everyone remembered it as a pivotal event in the campaign -- but it didn't generate that much media coverage, so the computer didn't see it as particularly important. I think it's an open question whether our memories are cloudy, or the computer's event detection is flawed.
If you're really into this stuff, here's a link to the paper. It's half baked, but feedback from the conference convinced me that the concept is worth pursuing as a research topic. Thanks again for your help!
Thursday, August 13, 2009
On putting faith in models
Here's a conversation I had with my brother via g-chat the other day. It's a great prototype of a conversation I've had many times in the last few months. The basic question is "how much faith can we place in mathematical models?" Most people seem skeptical; I'm more of a believer.
This particular exchange was unusual because 1) it was conveniently recorded, and 2) it went in some interesting and productive directions at the end. I'm posting it unedited, except for some spelling fixes and external links. Comments welcome.
This particular exchange was unusual because 1) it was conveniently recorded, and 2) it went in some interesting and productive directions at the end. I'm posting it unedited, except for some spelling fixes and external links. Comments welcome.
10:44 AM Sam: this is probably trivial compared to the analysis you usually look at, but i thought you might like it nonetheless
me: I'll check it out
| 6 minutes |
10:51 AM me: Interesting stuff
I hadn't seen the wine article before
10:52 AM I'd read the paper on war, but I hadn't seen the TED talk
10:54 AM This is exactly the kind of stuff I'm interested in doing
Sam: have you read when genius failed?
me: no
Sam: i can't remember if we talked about it already
it's about a hedgefund
10:55 AM and their story
it's the classic cautionary tale of putting too much faith in models
me: oh, yes
I haven't read it, but I know the gist
10:56 AM I would change the interpretation a little bit and say putting too much faith in a theory.
A model is a theory that happens to be expressed mathematically
Like any theory, if your assumptions are bad your conclusions will be bad
10:57 AM Sam: it was all mathematical
they determined 'actual' risk spreads based on reams of historical data and market conditions
10:58 AM and then tried to beat the market by playing the spreads dictated by their systems
me: right
as I understand it, the mistake they made was in the way they calculated risk
10:59 AM Sam: this is an interesting meta-argument
because you're pointing to the specific problem of their models
where i'm saying their mistake was over reliance on models
me: yes
11:00 AM this is a live debate in social science
I'm a pretty strong advocate for the quant side
11:01 AM Sam: hm
me: I'd say the key question is "is there anything substantively different about theories expressed in math verses theories expressed in English?"
Sam: this is probably a classic
me: classic?
11:02 AM Sam: academics vs business
11:03 AM me: hm
maybe
it's not so clean cut, though, because there are academics who reject the quant stuff and businessmen who embrace it
11:04 AM I think it has to do with the way people think about math
Is it a set of fixed processes for getting answers, or is it a language for expressing ideas?
11:05 AM If it's a language, then the fault for bad models lies is the assumptions expressed, not the language for expressing them.
| 5 minutes |
11:10 AM Sam: accepting that an omniscient agent could express all ideas as formulas and believing that you can are different, right?
11:11 AM its that leap where you decide to stake a business and millions of dollars on the formula that puts you on one side of the line or the other, in my opinion
me: yes, fair points
11:13 AM It seems to me that you're introducing another aspect of theories, which is that they don't just have to expressed, they also have to be acted upon
and strange things can happen when you act on a theory without fully understanding it
It's kind of a Jurassic Park idea
11:14 AM Sam: ha
i guess ultimately the moral to when genius is the same as jurassic park
11:15 AM but trust funds drying up is less dramatic than dino-carnage
me: so maybe the problem with models isn't that they are more likely to be wrong, but that they invite careless extrapolation
Sam: i don't think it's careless
it's hubris
me: "I'm going to leverage a billion dollars 30 times"
11:16 AM "I'm going to bring back dinosaurs from the dead, focusing mainly on intelligent top predators"
Sam: it's the opposite of being careless- it's spending so much time working on a model that you believe that you can and have thought of everything
11:17 AM its validating that model again and again against the datasets you have without allowing for future events to be unknowable
11:18 AM me: I agree -- my only reservation would be that hubris is not specific to people who frame their models using math
Sam: of course
the opposite story is much more common
which is why when genius failed is a story worth telling
11:19 AM when hubris failed would be too common
11:21 AM me: so instead of "people fail because of math," it's "even people using math can fail because of hubris"
11:22 AM Sam: yeah, that's the gist
me: okay, I don't have to feel so defensive now
11:23 AM a lot of the people around me here are model-builders
some of the faculty are among the best in the world
I see both types
11:24 AM Some use models for the sake of transparency -- all the assumptions are laid out for criticism and improvement
Others use models for the sake of beating up on people who don't speak game theory
11:25 AM I'm pretty invested in the notion that models can help the process of collective learning, and it frustrates me when the arrogant ones give modeling a bad name
Subscribe to:
Posts (Atom)
