What examples for successful prediction do we have in social science?

A question that's been bugging me: as social scientists, what can we predict reasonably well?

Following Kuhn's definition for a scientific paradigm, I'm focusing on:
1) social phenomena
2) that we can predict with enough accuracy to be useful
3) using technical skills that require special training.

I've found surprisingly few examples that satisfy all of these criteria. Only three, in fact.

1. Aptitude testing
   Prediction: How well will a person perform in school, a given job, the Army, etc.?
   Technical skill: Psychometrics
   


2. Microeconomics
   Prediction: What effect will various market interventions have on the price, quantity supplied, etc. for a specific good?
   Technical skill: Producer, consumer, and game theory
   


3. Election polling
   Prediction: Who will win in a given election?
   Technical skills: Survey design, sampling theory
   

Can you think of any others? I've got to be missing some.

What areas *should* we be able to predict? We have all kinds of new tools as social scientists. It seems like we should be ready to tackle some new challenges.

How to do research that has an impact, and still survive

Fascinating talk by Richard Hamming, of hamming code fame: You and Your Research. It's a very long and detailed discussion of how to build a scientific career in a way that will maximize the long-term impact of your work.

And here's an interesting counterpoint from the other end of a scientific career: how to survive grad school. Some Modest Advice for Graduate Students.

Glenn Beck conspiracy generator

A Glenn Beck conspiracy generator.

How does this thing work? I'm guessing mturk or some mailing list. The phrases don't seem quite formulaic enough for Markov generation or automated madlibs.

Best. Weather. Forecast. Ever.

Weatherspark uses the same data as everyone else, but they make it so much more usable. Gorgeous interactive maps, trends, and predictions.

Link mishmash

Here's some of what I've been reading/watching online lately.

1. From TED, on babies and how they "take statistics" to learn language. Super cute.

• Comprehension question: at what age did you stop being able to understand Mandarin?

• Presentation question: why does using FMRI/MEG/etc. increase one's credibility as a scientist? Several minutes of this 10-minute clip talk about brain activity and MEG output. But none of the important findings here are based on that data -- only speculation.

2. Via marginal revolution, a fair rent calculator.

This is for roommates deciding how much the big room with the extra window is worth. Based on this analysis, which uses surveys and hedonic regression to put numbers on preferences. (PS - "Hedonic Regression" would be a fantastic name for a nerd band.)


3. Two articles on credentialism in academia: from businessinsider, via Mike Bommarito; and from Steven Hsu, at Andrew Gelman's blog.

Interesting points made here. Has the value of formal, certified education increased or decreased? What about the quality of education? I've grown increasingly pessimistic about this topic of late.

4. And last, but not least a caption contest at wondermark (I love this web comic). Here's the pic. Submissions wanted.

How to win the final wager in Jeopardy

Here's a fun sideline to the human-vs-AI Jeopardy showdown: optimal wagering strategies for final Jeopardy. The challenge: given what you know about yourself, your opponents, their bankrolls, and the question, choose the betting strategy most likely to secure a win or at least a tie. What do you do?

As it happens, some smart game theorists have already worked on this problem. Work in the 1990's shows that against your average human, it's not too complicated.

• If you're winning, assume your opponents will bet everything, and react accordingly by betting enough to double the number two player's score, plus one dollar.
• If you're in the middle it can be tricky, but a good bet is to triple your score and subtract double the first player's score.
• If you're losing by a lot, bet everything.

Following these rules should increase your chances of winning, regardless of your starting position. (See pg 271 in the paper.)

Against a human (or computer) who knows some game theory, the ideal strategy is complicated, because your strategizing will prompt strategizing by your opponents, and vice versa, ad infinitum. The Gilbert and Hatcher paper gives a solution for the two-player scenario. This situation happens when one contestant has no money to wager in the final round -- it's actually not all that unusual.

The three-player scenario is unsolved, but it's almost certainly a mixed strategy. Sounds like a good problem for a different kind of AI: computational game theory. Given that simple versions of poker have been completely solved, a mixed strategy over wagers should be child's play.

With all that as background, here's IBM's page on how Watson wagers. They cite the ridiculously detailed Jeopardy folklore of betting strategies --- the entries for "two-thirds" and "Shore's conjecture" are particularly on point --- but evidently, they haven't worked through the game theory. Knowing this, Ken and Brad should be able to outflank Watson in the final round, giving them slightly better odds at cinching a win.

Punchline: in the world of AI, Watson is a verbal genius in need of some remedial math.

Realtime Congress API

Just ran across this new API, for Congress, in real time. Put out by the Sunlight Foundation, an organization using "cutting-edge technology and ideas to make government transparent and accountable," the API allows you to track, floor actions, video, committee hearings, bills, amendments, votes, and other documents in real time.

Really looking forward to playing with this.

Hard problems in social science

A Harvard conference on "Hard problems in social science" just wrapped up, with voting on the most important and most difficult problems. The ran the poll on facebook with 10,000 people weighing in.

Frontrunners include:

• World peace
• Sustainable population growth
• Collective decision making
• Stable institutions
  

The pdf with survey results is here. Videos of speakers are here. Very interesting stuff

Cracking the Ontario lottery - via Wired

A great story on the statistician who cracked the Ontario lottery -- how he cracked it, why the lottery didn't believe him, how other lotteries are probably breakable as well, and why the mob likes it that way. A great read, all the way through.

Model Makers' Hippocratic Oath

Model Makers' Hippocratic Oath:

• I will remember that I didn't make the world and that it doesn't satisfy my equations.


• Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.


• I will never sacrifice reality for elegance without explaining why I have done so. Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.


• I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.

Via Statistical Modeling, Causal Inference, and Social Science