A graphic design challenge...

Next semester, I'm going to be conducting a large-scale survey of bloggers for my dissertation. (I've had a pilot wave in the field for the last several weeks.) For credibility's sake, I need a public face for the project -- a web site people can visit to reassure themselves that my survey request isn't some kind of phishing scam.

I'm branding the project as the Online Political Speech Project, with the goal of "understanding the voices of the Internet." I've worked out some rough ideas for visual motifs, but I'm no graphic designer. If you are, or know someone who is, I'd love to be in touch. I'm hoping this can be a fun project for some talented graphic designer...



Project specs: I need to design and lay out this web page by the second week in January. I'm imagining about five pages (splash page, FAQ, contact, blog, data repository). Scope of work would be limited to designing templates -- I can plug in content later.

This should be reasonably quick work, with a lot of creative control for the designer. I can offer a little cash, but probably a lot less than going professional rates. (That's because, as a grad student, I get paid a lot less than going professional rates, too.) Maybe a good portfolio piece for a student web designer...? Or else a fun project for someone who wants to play with some of the styles and themes that have emerged lately on the web...?

Please email me (agong at umich dot edu) if you're interested.

Google's n-gram viewer - pros and cons

Google just released an n-gram viewing tool for tracking trends in half a trillion words across 400 years of text. Check it out. It's super fast and interesting, kind of like Google trends on timescale steroids. Even better, they released the data.

Assorted thoughts poached from conversations with friends:

• Someone should tell Fox about "Christmas"

• I predict that in the year 165166 AD, fully one hundred percent of the words written in English will be the word "dance." (This will make 500-word APSA applications much more straightforward.)

My take
(In 18 words: For scientists, a small, representative sample usually beats a large, non-random sample. Google has given us the latter.)

The n-gram viewer will be a lot of fun for journalists, who often want a quick graph to illustrate a trend. It's a gold mine for linguists, who want to understand how syntax and semantics have changed over time. For both of these fields, lots of data equals better.

As a bonus for me, maybe this tool will popularize the term n-gram, so I won't have to keep explaining it in conference talks.

I'm not sure about the impact the n-gram viewer will have on the areas where the researchers seem to want to apply it most: culture, sociology, and social science in general. The reason is that in those fields we tend to care a lot about who is speaking, not just how much speech we have on record. This is why the field of sampling theory has been a consistent cornerstone in social science for almost a century. From the press releases, we can see Google's claim that this is 5.2% of all books ever written. But we don't know if it's a random sample.

And in any case, books have never been written by a random sample of people. I suspect that the n-gram viewer will have very little to offer to researchers hoping to study the culture and sociology of, say, blacks in the Jim Crow south, or the early phases of Lenin's revolution. By and large, we already know what cultural elites were saying in those periods, and the rest weren't writing books.

This means that there are two filters in place for any social scientist who wants to pull cultural trends out of these data. First, there's Google's method for sampling books, for which I haven't seen any documentation. (No offense, but this is pretty typical of computer scientists, who think in terms of scale more than sampling). Second, there's the authorship filter: you have to keep in mind that any trends are derived from written language, produced by whatever subset of the population was literate in a given period

Example
As a political scientist, I'm interested in conflict. If you go to the site and punch in "war," from 1500 to 2000, you get a graph showing quite a few interesting trends. Here are some stories that I could naively tell from this data. I suspect that many are false.

• In general, humanity has been far more interested in war since 1750.
• That said, interest in war has generally declined since then.
• Interest in wars spikes during wars.
• Proportionally, wars since 1750 affect people far less than they did previously -- the use of the term "war" jumped up to 5 times the period baseline during the English civil war, but less than double the period baseline during the World Wars.
• World wars I and II were the historical high points for human interests in war, but only slightly higher than the sustained interest of the peak period in 1750.

Interesting conjectures, and I can spin a story for each of these. In several cases, I'm pretty sure we're seeing the effects of cultural or linguistic artifacts in the data. Potential confounders include: changes in literacy rates, introduction of public schooling, the American revolution, improvements in printing technology, linguistic drift, etc.

I'm cautiously pessimistic here. Given all the potential confounding variables, it's hard to see how we can sort out what's really going on in the line graphs. Maybe we can do it with fancy statistics. But we're not there yet.

I'll give the last word to BJP:

My instinct suggests that what is talking here is less the folds of culture and more the wrinkles of the record of the technology expressing that culture.

Link mish-mash

All kinds of interesting stuff on the web today. Not that this wasn't also true yesterday, or the day before...

• IBM says its Watson AI is ready to take on human Jeopardy champs for a $1 million prize. The showdown is scheduled for Feb 14. This is reminiscent of the Kasparov/Deep Blue showdown, except that Watson will be competing on human home turf: making sense of the linguistic ambiguity in the hints, phrasing, puns, etc. of Jeopardy prompts. (The AI has one advantage: I bet Watson will always remember to phrase its answer in the form of a question.)

• A rehash of physicist Aaron Clauset's work on "the physics of terrorism." I'm not a big fan of his stuff, to be honest. My view: Clauset showed that the severity of terrorist attacks follow a powerlaw distribution, and has been wildly extrapolating from that single finding ever since.

• A Jeremiad about the state of journalism, from Pulitzer prize winner David Cay Johnson. He talks trends (reporters know less and less about government; papers keep cutting content and raising prices), and hints at causes. The last paragraph is especially intriguing to me. Read it as a claim about how good reporting is supposed to uncover truth.

• Predictions about 2011 from 1931. Eighty years ago, the NYTimes gathered a brain trust of experts in various fields and asked them what the world would look like today. Follow the link for predictions and some commentary. (Hat tip Marginal Revolution.)

• This paper should depress my libertarian friends. Evidently, profit is evil, with an r-value of -.62.

• The best data visualization projects of 2010, from FlowingData. Beautiful and informative.
  

HPN health prize: $3,000,000

The Heritage Provider Network is offering a $3 million prize for predicting hospitalization -- which patients are likely to be hospitalized or rehospitalized?

In 2006 well over $30 billion was spent on unnecessary hospital admissions. How many of those hospital admissions could have been avoided if only we had real-time information as to which patients were at risk for future hospitalization?

Programming Language Popularity Contest

I keep telling people that python is the language to learn. Here's proof. Half a million github and StackOverflow programmers can't be wrong, right?

(Ignore javascript. It's an in-browser language only. Fun if you want to animate a web page; useless if you want to write a file.)



via : Programming Language Popularity Contest

Journalism and activism: Stylized facts

Trying this idea on for size. Reactions?

(Also, I'm attempting to post to facebook via Notes. Let's hope the feed actually works now...)

The United States of Autocomplete

Hat tip FlowingData: The United States of Autocomplete

Very Small Array has some fun Google's autocomplete. Utah... Jazz. Kentucky... Fried Chicken. New York... Times.

[Very Small Array via @mericson]

Arsenic-based life and autocatalytic sets

NASA has just announced the discovery of a new kind of extremophile bacteria. If I'm reading the (sketchy) early release info correctly, these critters use arsenic in their DNA, instead of phosphorus. This is a biggish deal because every other known life form uses phosphorus.

How big a deal? Setting aside shades of the Andromeda Strain*, people seem pretty underwhelmed. Partly it's a bait-and-switch involving the NASA brand: the microbe in question, Gammaproteobacteria GFAJ-1, isn't an alien visitor. It's a mutant strain of a run-of-the-mill terrestrial bug.

So the real question is: How surprised should we be that little GFAJ-1 managed to assimilate arsenic into its DNA? Or the converse: if it's so easy, should we be surprised that no other life form bothered to do the same thing?

Taking a conversational step sideways, this seems like a good moment to put in a plug for a really fascinating theory on the origin of life: autocatalytic sets. This theory--which I find persuasive--argues that life isn't rare or unexpected -- it's virtually inevitable. I can't find a good non-technical description of this underappreciated set of ideas online, so I thought I'd take a crack at it here . (My explanation is based largely on the opening chapters of Kauffman's At Home in the Universe).

The puzzle: all forms of life we know of are pretty complicated. There are a couple hundred cell differentiations in a human body; thousands of enzymes in a working cell; millions of base pairs in even the simplest DNA. (Prions and viruses, which are often simpler, don't count because they can't replicate without hijacking a cell's machinery.) With these kinds of numbers, the odds of getting the just right combination for life are astronomical. Try flipping 220 million coins until they line up with the base pairs in a single human chromosome. Practically speaking, it will never happen.

For most explanations for the origins of life, complexity is a stumbling block. We just got lucky, or God intervened, or maybe an asteroid seeded the planet to produce something capable of self-replication. There's a leap in the logic at the point where we try to explain the incredibly low probability of life emerging spontaneously from a lifeless chemical soup.

The autocatalytic theory turns this logic around: it argues that life exists because of its chemical complexity, not in spite of it.

The theory builds from three simple assumptions. First, some chemicals react; most don't, with roughly constant odds for any given pair of chemicals reacting. Second, chemical reactions produce new chemicals. Third, the number of possible chemicals is very large.

Thought experiment: suppose you find yourself in front of chemistry set containing a rack of n beakers, each filled with an unknown chemicals. Channeling your inner mad scientist, you begin mixing them at random. Suppose 1 in 100 pairs creates a reaction. How many total reactions should you be able to get out of your chemistry set?

If you have two chemicals, the odds of a reaction are just 1 in 100.

If you have three chemicals, you have three potential pairs, and the odds of a reaction are about 3 in 100.

If you have four chemicals, you actually have six potential pairs, so the odds of a reaction are a little better than 6 in 100.

At this point, exponentials start working rapidly in your favor. With five chemicals, you have 10 potential pairs, for a 9.5% chance of at least one reaction. Twelve chemicals gets you 66 pairs with 48.4% odds of at least one reaction. The deluxe 30-chemical starter kit has 435 potential pairs, with 98.7% odds of at least one reaction.

What does this prove? The number of likely reactions in a pool increases faster than the number of chemicals in the pool.

It keeps going, too. With 1 in 100 odds, you would expect to get about 4 reactions out of your 30-chemical kit. If each reaction creates a new chemical, you now have 34 chemicals in your pool, with correspondingly greater odds of additional reactions. Eventually, you pass a tipping point, and the expected number of compounds becomes infinite. If I've got my math right, it happens around 80 chemicals in this scenario, because the expected number of new reactions exceeds the number of reactions in the existing set. The more you mix, the more potential mixtures you get.

A quick pause: when we talk about chemicals, we're not talking about atoms in the periodic table of the elements. Except during fission and fusion, atoms themselves don't combine and react. Instead, we're talking about molecules.

In particular, organic molecules -- the ones that ended up supporting life -- fit this model very well. Enzymes, RNA, and DNA are all organic molecules. Believe it or not, a strand of DNA is one enormous molecule. Organic molecules are all built mainly from the same base atoms: carbon, hydrogen, oxygen, nitrogen, phosphorus -- and now arsenic(!) These atoms happen to be good at linking to form long chains. Because of the way these chains fold, they react with each other in often unpredictable ways. Most organic molecules don't react with each other, but quite a few do. And because the chains can get very long, the set of potential molecules made of CHONP atoms is essentially infinite.

Now getting back to the main story... We're halfway through. We've shown how simple rules for reactions can get you from a small-ish starting set to an infinite variety of chemicals to play with. It seems very reasonable to suppose that the primordial organic soup included enough organic reactants to pass the tipping point into infinite variety. But that just means a more flavorful soup. How do we get to life?

Setting aside the transcendental, life is defined by sustainable reproduction. A cell is a bag of chemicals and reactions that keeps working long enough to make at least one copy of itself. As part of the deal, the cell has to be in the right sort of environment, with whatever energy sources and nutrients are necessary.

Our cells achieve sustainability by using enzymes to catalyze other reactions. It turns out that the same logic that applies to pairs of reactions also applies to catalyzers: the probability of catalysts in a pool increases faster than the number of chemicals in the pool. Once you get enough chemicals, it's virtually certain that you'll have quite a few catalyzed reactions.

Here's the really nifty bit. As the number of catalyzed reactions in the set increases, eventually some of them will form an autocatalytic set -- a loop of reactions catalyzing each other. Reaction A creates the catalyst that enables reaction B, creating the catalyst for C, and so on back to catalyst Z that enables reaction A.

Based on the same logic we saw earlier, these loops always appear once the pool of chemicals gets large enough. They are typically long and complicated, cycling through a seemingly random group of chemicals among a much larger set of nutrients and byproducts. They tap nutrients and energy sources in the environment, increasing themselves the longer they run. In other words, autocatalytic sets look a whole lot like life as we know it.

I find this theory compelling. It takes the biggest objection to prevailing theories -- the inherent complexity of life -- and makes it the cornerstone of a different approach.

And as a bonus, it makes arsenic-based life forms seem very plausible. Given NASA's results, it seems reasonable to say that arsenic-based DNA is another unexplored evolutionary path for viable autocatalytic sets. Bill Bryson says it well in A Short History of Nearly Everything:

"It may be that the things that make [Earth] so splendid to us---well-proportioned Sun, doting Moon, sociable carbon, more molten magma than you can shake a stick at and all the rest---seem splendid simply because they are what we were born no count on. No one can altogether say. ... Other worlds may harbor beings thankful for their silvery lakes of mercury and drifting clouds of ammonia."

PS: Autocatalytic sets don't have much to do with the evolution/intelligent design debate. They propose a mechanism that could be responsible for jumpstarting evolution. So if you're comfortable with the idea that God would choose cosmological constants and direct evolutionary processes with some goal in mind, it probably won't bother you to add the idea that he would use chemical soups and catalysis networks along the way.

PPS: The main difference between an autocatalytic set and life as we know it is the absence of a cell wall. It's not hard to close the gap conceptually. Once a catalytic loop gets started, other loops usually form as well. At this point, competition and natural selection between autocatalytic cycles can kick in. If one autocatalytic loop happened to produce a hydrophobic byproduct (like a fat or lipid), it could easily act as a primitive cell wall. This kind of barrier would enable the autocatalytic loop to increase its concentration, and therefore its reaction rate. This kind of pseudocell would reproduce faster and very likely evolve into more sophisticated organisms.

*A ten word review of the Andromeda Strain: Typical Crichton--some interesting ideas; mildly annoying narration; mostly plotless.

** A wonderful book for putting some color on the messy process of scientific discovery.

Prezi - A quick review

I spent a couple hours playing around with Prezi, an online presentation builder being promoted as an alternative to the slideshow format of powerpoint, keynote, etc. Instead of a series of slides, Prezi presentations consist of a series of views over one large image. The format parallels drawing on a whiteboard, instead of clicking through slides on a projector. A good concept, but the execution is a little clunky.

My review: Preparing good presentations is time consuming, for two reasons: 1) it takes some trial and error to figure out the best way to express an idea, verbally and visually, and 2) presentation software is clunky, requiring a lot of fiddling to get things right. In my experience, spending time on (1) is fun and creative; spending time on (2) is frustrating and stressful.

Because of the "whiteboard" metaphor and brand emphasis on good design, I was hoping Prezi would deliver a slick and streamlined user experience. Being free from interface hassles and able focus on creative expression would be wonderful. Alas, I quickly ran into many GUI annoyances.

• The interface for importing images is very clunky. You have to download or save the image to your desktop, then upload. On the plus side, you can batch upload several images at a time.
• The whole image is static, which means that you can't mark up images over the course of a presentation. To some extent this makes sense -- dynamic images would mess up the concept of arranging your display in space rather than time. However, it breaks the whiteboard metaphor. When I do whiteboard presentations, I often have an agenda that I revisit, adding checkmarks and lines to relevant content. I can't do that in Prezi.
  
• Rudimentary tools for grouping object are not available. This one really gets me. You can accomplish the same thing (visually) by putting several objects together in an invisible frame. But every time you want to move the group, it takes several extra clicks to select everything and drag it around. Poor usability.
• You can only use a handful of presentation styles. Your only alternative is to hire Prezi staff to build a custom style for $450.

Summary: I would really like a tool that lets me express myself clearly, fast. Prezi offers some advantages for clarity, but not really for speed. Overall, I'm mildly impressed, but not overwhelmed. For the moment, the main benefit of Prezi seems to be novelty.

Forums and platforms for crowdsourcing

I've always thought of crowdsourcing as a narrow, niche thing. But then I ran across this excellent list of forums and platforms for crowdsourcing. Skimming down the list, I realized that crowdsourcing is actually starting to play an important role in quite a few industries.

It looks like we're just starting to climb the adoption curve here. Where should we expect crowdsourcing to stop? What sorts of problems will it solve (and not solve)? Who won't ever use it?

PS: In case you haven't run across the term before, here's wikipedia's definition of crowdsourcing:

Crowdsourcing is the act of outsourcing tasks, traditionally performed by an employee or contractor, to an undefined, large group of people or community (a crowd), through an open call.

How long are books in political science?

As a polisci student who has passed his prelims, I have a very good idea how long books take to read. But I realized this week that I didn't have a good idea how long books take to write.

As I was setting up the template to write a book-format dissertation, I decided to give it a look. Here are some rough stats for five well-known polisci books*, pulled from my reading shelf more or less at random:

	Hindman	Zaller	Mutz	Lupia and McCubbins	Huber and Shipan
Lines/page	38	45	35	42	39
Words/line	13	13	11	12	11
Total pages	141	310	125	229	210
Total chapters	7	12	5	10	8
~Pages/chapter	20.1	25.8	25	22.9	26.2
~Total words	69,654	18,1350	48,125	11,5416	90,090

These estimates are probably biased upwards, because I based them on pages of full text without any tables, graphs, or chapter breaks. But still, it's a pretty good idea for the rough scale of the project.

To put it in perspective, the current working draft for my dissertation includes 10,653 words. If I'm shooting for 70,000 total, that means I'm 15% done already!


*These are the books I used for estimates:
Matthew Hindman, The Myth of Digital Democracy
John Zaller, The Nature and Origins of Mass Opinion
Diana Mutz, Hearing the Other Side
Arthur Lupia and Matthew McCubbins, The Democratic Dilemma
John Huber and Chuck Shipan, Deliberate Discretion

An update on voice recognition software

Thanks to everyone who sent comments about voice recognition via Dragon, Microsoft, and Mac. I'm still playing with different options to see what will work.

In the meantime, I've discovered another option: Google Voice. It turns out there's an easy way to set up Google Voice so that you can dictate messages to yourself (info here). GV will then automatically transcribe them and send them to your email, or mobile device. Nifty!

Since it's free and didn't require any software installation or training, I decided to give this a shot first. The result: laughably bad transcription (see below), but a substantial boost in productivity. How's that? For me, drafting is the most time-consuming part of writing. Once I have some basic ideas on paper, I can edit and elaborate reasonably quickly. But it takes me a long time to get that first version out.

As a result, dictating an early version has been very helpful. It forces me to say something. Since I'll be editing soon, it doesn't matter if that something is bad (it is, usually). So dictation, even with GV's horribly inaccurate transcription has worked pretty well for me. Bottom line: I'm certainly going to be investing in a voice recognition package in the near future.

An example of GV transcription. Here's what I said:

This chapter describes methods and data sources for the book. My goal is to describe the logic behind the research design. The focus here is validity: what types of conclusions can we draw from these data? Technical details---of which there are many---are saved for appendices.

Here's what Google thought I said:

This chapter just tries messages and data sources for the book. My goal is to describe the logic behind the research design. The focus here is validity. What types of conclusions. Can we draw from the Yeah, technical details. I wish there are many her saved for the appendicitis.

In fairness, Google just got into this business in the last year or so. I'm sure their transcription will get better over time. But for the moment, "I wish there are many her saved for the appendicitis."

Anybody have any experience with dictation software?

Faced with writing a 200-page thesis in the next year, I'm toying with the idea of getting myself a voice recognition software package. Dragon seems to be the industry leader. Here's a demo.

The software seems to work, is getting good reviews on Amazon, and is not terribly expensive. On the other hand, it seems like it might be hard to find a place to talk loudly to oneself and a computer for long periods of time. It also seems likely that training the software and editing its mistakes could be pretty time consuming. Also, Dragon seems built with MS Office in mind. Given that I lean open source (and don't even have Office on my laptop), would it work for me?

Bottom line: would this be more like the digital stylus I got a few months ago (and use all the time)? Or the extra external hard drives I got a year before that (and have only booted up to make sure they work)?


So I'm on the fence. I really wish Dragon had a trial version. Anybody have anything to add here?

PS: Yes, I do know that one episode of The Office. (Dwight: "Cancel card. Can-suhl car-Duh.")

PPS: How long do you think until a viable open-source option opens up? Probably at least a couple years, unless someone like Google decides to launch a free version. They seem to be making moves in that direct (see here for an easy application, and here for a broader one).

Training a text classifier

I've been doing a lot of work with text classifiers lately. (See this post for an example.) It's an interesting process blending intuition, text, and math in some nifty ways.

Anyway, I was going to spell out how it all works in a blog post, but I got side tracked making an illustration, and now I'm out of time for blogging. But I like the illustration. More to come...

TED talk: Teaching kids real math with computers | Video on TED.com

Conrad Wolfram: Teaching kids real math with computers | Video on TED.com

I like this.

Starting to get some dissertation results...

Apologies for the long delay between posts. Stock excuse: "Dissertation... blah blah blah..."

Actually, I'm starting to get some nifty results from my dissertation. I've spent a long summer writing surveys and software, and in the next few weeks I hope to have something to show for it. Exhibit A: a word cloud for an automated classifier of political content.


Orange words are associated with political content, and blue words are disassociated. The size of a word denotes the strength of association -- essentially, the size of each word corresponds to the absolute value of the beta value of the word in a logistic regression with "political-ness" as the dependent variable. The layout of the words is done by computer algorithm to conserve space; it doesn't carry any important information.

I used wordle for the layout. The classifier runs regularized logistic regression using the scikits.learn package for python. The training data is from a team of undergraduate research assistants.

Daily Beast's Election Oracle

Here's a nifty new application in sentiment analysis: the Daily Beast's "Election Oracle." It's based on an engine called WiseWindow, which apparently scrapes the Web in real time, searching for content relevant to all the midterm elections, then makes predictions. One interesting touch is that DailyBeast pipes in RealClearPolitics' latest poll numbers, inviting head-to-head comparisons.

If the screen-scraping works the way it's intended, you would expect WiseWindow's numbers to change a little faster than the polls, since polls are only conducted every so often, and usually include a rolling average.

I'm skeptical. Not because I don't believe that it can be done, but because I've read up on the top-of-the-line opinion mining algorithms, and they're still full of holes. Computer scientists are smart people, but they haven't absorbed the lessons that pollsters have spent almost 100 years learning.

My particular brand of procrastination

On procrastination

"Victor Hugo would write naked and tell his valet to hide his clothes so that he’d be unable to go outside when he was supposed to be writing."

If only I didn't have to do so much work from the CAP lab...

Here's a great, well-sourced article in the New Yorker on the psychology, philosophy, economics, and history of procrastination. Worth a read for all grad students.

A minute to learn, a lifetime to master

My family was in town last weekend (new baby--very exciting) and my brother Sam and I got to talking about board games, and what makes a good game. One reasonable standard comes from Othello's tagline: "a minute to learn, a lifetime to master." For strategy games, at least, we like the idea of complex gameplay emerging from simple rules.

Following that idea, we pulled up a list of ~30 well-known games and rated them for "game complexity" (How complicated are the rules?) and "strategy complexity" (How complicated are strategies for winning?). We came up with our ratings separately, but they were strongly correlated (R-squared values of .68 and .73, respectively).

The strength of this correlation surprised me at first. In retrospect, board games have been such an important part of brotherly bonding that I shouldn't have been taken aback. Case in point: we had agreed to use 1 to 10 scales, but we both independently decided that War and Candyland should score zero for strategy complexity, because neither game ever presents players with a choice.

With data in hand, I averaged our scores and then plotted game complexity against strategy complexity. The two dimensions of complexity are correlated (R2 = .494). Note that both my favorite game (Go), and his favorite game (Texas holdem) are strong positive outliers.

Apologies for the ugly graph. If I had the time, I'd have plugged in logos for all the games. Still, the picture mostly comes through.

Link mishmash

Interesting post by Dan Hopkins on the Monkey Cage: Spin and the economy -- Can newspapers convince people the economy is better or worse than it really is? No.

Also from the MC: predictions for the 2010 elections. We're short of complete consensus, but the Dems are likely to lose the House.

From FlowingData: a beautiful delayed-shutter picture of fireflies in a forest. See also the delayed-shutter picture of a Roomba.

From NYTimes: folk wisdom about study habits -- some of it masquerading as scientific -- is often dead wrong. This whole article is worth reading, but let me spoil a few of the punch lines. Designated study rooms -- bad. (What about designated study times?) Learning styles (e.g. "I'm a visual learner") -- very little supporting evidence. Testing -- often beneficial.

An xkcd wedding cake. I love this web comic.

Please link/subscribe to me!

A cordial request: if you read or feed this blog, please link, subscribe, etc. in whatever format is right for you. I'd like to bump up my PageRank etc. a little. I promise I won't do this often.

Thank you! Here's a puppy:


We now return to our regularly scheduled social/computer-sciency blogging.

Probabilistic computing?

Hat tip SMCISS.

For over 60 years, computers have been based on digital computing principles. Data is represented as bits (0s and 1s). Boolean logic gates perform operations on these bits. A processor steps through many of these operations serially in order to perform a function. However, today’s most interesting problems are not at all suited to this approach.

Here at Lyric Semiconductor, we are redesigning information processing circuits from the ground up to natively process probabilities: from the gate circuits to the processor architecture to the programming language. As a result, many applications that today require a thousand conventional processors will soon run in just one Lyric processor, providing 1,000x efficiencies in cost, power, and size.

Hat tip SMCISS.

Psychologists say that power corrupts, via WSJ

Psychologists say that power corrupts, via WSJ. I know that this thesis is pretty widely accepted, but I want to snipe at it. I think we may be mistaking correlation for causation. The general arc of my argument is that thuggishness may (sometimes) lead to power, instead of the other way around.

First, all the cited psych studies showing "nice guys finish first" are from areas with dense social networks (dorms, sororities, chimp clans) and repeated person-to-person interactions. Not surprisingly, likability translates to popularity here. But I don't see any particular reason why that conclusion should translate to accumulation of power through job interviews or elections. I'm going out on a limb here, but perhaps the qualities that make frat boys popular are not the same qualities that put CEOs in power.

Second, at risk of alienating a lot of my friends in political psychology, I'm not a big fan of gimmicky priming studies. For instance: tell me about a time when you felt powerful, then write the letter E on your head. You wrote it backwards? See -- power corrupts.

Err... run that by me again? At what point does this experiment measure "power," and "corruption," and where does it demonstrate a casual relationship between the two? Sure, the researchers who did this study can do a song and dance about it, but at the end of the day, there are a lot of leaps in the logic. The study is gimmicky. Interesting maybe, but nowhere close to conclusive.

I completely accept the notion that our minds move between different mental states and that primed manipulations can nudge us from one state to another. But I'm not persuaded that we've mapped the set of states and priming cues in enough detail to draw general conclusions -- particularly when treatments and responses are measured in ways that never occur in normal human experience.

Third, the idea that transparency might fix things is nice, but lacking evidence. This is a place where careful study of incentives and strategy could add a lot of leverage. Go game theory!

Parting shot: I'm not anti-niceness or pro-corruption, but I think we often demonize authority and authority figures. There are an awful lot of social problems that best resolved through the benevolent use of authority, and I know plenty of people who live up to that charge. To the extent that power has a corrupting tendency, we should be looking for, promoting, and celebrating those who are proof against it.

Wacom tablet: post-purchase rationalization

So I bought a Wacom Bamboo tablet last week. It's a nice digital pen tablet, with a drawing area about the size of a postcard -- a good size for sketching & gesture-based interfaces. It plugs into a USB port and (after some fiddling with the drivers) works very nicely with Windows, Firefox, Inkscape, etc.

That said, I'm not really sure why I bought the thing. I do a lot of writing and programming, which leads to a lot of typing, but not much clicking and dragging. Sure, the touchpad on my notebook is small, but it's not really $60 small. I blame postpartum sleep deprivation. It might also have something to do with watching this TED talk.

Anyway, in an effort to assuage my post-purchase cognitive dissonance, I've been telling myself that if I can improve the speed and accuracy of my clicks by just a fraction, this tablet thing will easily pay itself off in productivity in the long run. Right?

To bring some proof to that claim, I dug up this flash-based target practice game. Little targets fly around the screen and you try to click on them: score points for every target you hit; bonus points for consecutive bulls-eyes; miss too many times and the game is over. Great sound effects. This is high brainpower stuff.

I played three rounds in time trial mode, using the touchpad. Mean score 438. Then I played thrice with the tablet. Mean score 685!

To make sure this wasn't just an improvement in my reflexes and strategy (shoot the spring targets at the apex; don't waste a shot on a yo-yo target that might be yanked), I employed an interrupted time series design and played six more rounds. Mean score with the touchpad: 407. Mean score with the tablet: 977!

With that kind of performance improvement, the tablet was clearly worth it. Minority report, here I come.

TED talk on origami

James Lang, physicist and origami master, speaks at TED. He talks about a fascinating intersection of art and math: origami.

This is a great, visual talk that links ideas you would never expect to come together: water bombs, ibex, and spiders turn into satellites, heart valves, and airbags, using the math of trees and circle packing. A really great example of the unexpected practical value of mathematics.

Hat tip to my brother Matthew for putting me onto Lang's work.

P != NP? The biggest mathematical problem in computer science may be solved

Banks, businesses, and credit card users can rest easy: public key encryption is still secure. Mathematicians breathe a sigh of relief: computers aren't going to replace you by churning out brute force proofs. Vinay Deolalikar, a researcher at HP Labs, is claiming that he has verified the most important conjecture in mathematical computer science: P != NP.

Some links, and then onto an explanation of why this is all a big deal.

• Greg Baker broke the story.
• The slashdot squib.
• Here's a very mathy site with some good context and historical background.
• Here's a wiki section on consequences of a proof one way or the other. I don't like it much, so I'm going to try to do better here.

What's the difference between P and NP?
Suppose you have a friend -- a real whiz at math and logic. "I can answer any question you ask*," she promises, "it just might take some time." Before you ask her to solve your sudoku, route the world's airline traffic, or crack your bank's electronic encryption key, it's worth figuring out "Just how long is it going to take?"

For many kinds of problems, the answer depends on how many moving parts there are. To cite a famous example, suppose I'm a traveling salesman, using planes, trains, and automobiles to get between cities. I have n cities to visit. What is the fastest route that lets me visit every city?

Intuitively, the more cities there are, the harder this problem is going to be. If n=1, the answer is trivial. I start in the single city and just stay there -- no travel time. If n=2, the answer is easy. It's equal to the amount of time between the two cities (or double that, if I'm required to make a round trip.) If n=3, it starts to get complicated because I have several paths to check. If n=100, then solving the problem could take a long time. Even your math whiz friend might have to chew on it for a while.

In the world of computer science, P-type problems are the fast, easy, tractable ones. They can be solved in "polynomial time": if the problem has more moving parts, it will take longer to solve, but not a whole lot longer. Mathematically speaking, a problem belongs to complexity class P if you can write a function f(n) = n^k and prove that a problem with n elements can be solved in f(n) operations or less. Constructing proofs of this kind is a big part of what mathematical computer scientists do.

If we could prove that k=3 for the traveling salesman problem -- we can't, by the way -- then we could guarantee that a computer could solve the 100-city problem in 1 million time steps or less. If your math whiz friend uses an Intel chip running millions of operations per second, that doesn't look so bad. For P problems, we can put a bound on running time, and know that as n increases, the performance bound increases at worst by the exponent k.

NP-type problems can get harder, because the definition is looser. Instead of requiring your friend to come up with the answer in polynomial time, we only require her to verify an answer in polynomial time. ("My boss says I should go to Albuquerque, then Boston, then Chicago, then Denver. Is that really the fastest route?" "Hang on. I'll get back to you in n^k seconds.") If a problem is of type NP, then we can find the right answer using a "brute force," guess-and-check strategy. Unfortunately, as the number of moving parts increases, the number of possible answers explodes, and the the time required to solve the problem can get a lot longer.

Finding the solution to NP problems when n is large can take a ridiculously long time. Some NP problems are so computationally intensive that you could take all the computers in the world and multiply their processing power by a billion, run them in parallel for the lifespan of the universe, and still not find the solution. Problems or this type are perfectly solvable, but too complicated to be solved perfectly.

There's a third class of problems, called "NP-hard," that are even worse. Things are bad enough already that I'm not going to get into them here. You may also hear about "NP-complete" problems. These are problems that are both NP and NP-hard. (See this very helpful Venn diagram from wikipedia.)



The P !=NP Conjecture
But there's an intriguing catch. Until last week, nobody was sure whether any truly NP problems existed.** Maybe, just maybe, some mathematical or computational trick would allow us to convert tricky NP problems into much easier P problems.

Most computer scientists didn't believe that this was possible -- but they weren't sure. Thus, P!=NP was a longstanding conjecture in the field. (By the way, "!=" means "does not equal" in many programming languages). Many proofs had been attempted to show that P=NP, or P!=NP, but all of them had failed. In 2000, the Clay Mathematics Institute posed P!=NP as one of seven of the most important unsolved problems in mathematics. They offered a million dollar prize for a proof or refutation of the conjecture. The prize has gone unclaimed for a decade.

Last week, Vinay Deolalikar (cautiously) began circulating a 100-page paper claiming to prove that P!=NP. His proof hasn't been published or confirmed yet, but if he's right, he's solved one of the great open problems in mathematics.

So what?
Let's assume for the moment that Deolalikar has dotted his i's and crossed his t's. What would a P!=NP proof mean?

One immediate implication is in code-breaking. Public key encryption -- the basis for most online banking, e-commerce, and secure file transfers -- relies on the assumption that certain mathematical problems are more than P-difficult. Until last week, that was just an assumption. There was always the possibility that someone would crack the P-NP problem and be able to break these codes easily. A P!=NP proof would eliminate that threat. (Other threats to security could still come from number theory or quantum computing.) For wikileaks, credit card companies, and video pirates, this is reason to celebrate.

On the downside, a proof would put a computationally cheap solution to the traveling salesman problem (and every other NP-but-not-P problem) permanently out of reach. This includes a host of practical problems in logistics, engineering, supply chain management, communications, and network theory. Would-be optimizers in those areas will sigh, accept some slack in their systems, and continue looking for "good enough" solutions and heuristics. Expect a little more attention to clever approximations, parallel processing, and statistical approaches to problem solving.

A P!=NP proof can also help explain the messiness and confusion of politics and history. Recent work in game theory demonstrates that finding stable solutions in strategic interactions with multiple players (e.g. solving a game) is a computationally difficult problem. If it turns out that reaching equilibrium is NP, that would put bounds on the quality of likely policy solutions. For instance, we can pose the problem of writing legislation and building a coalition to ratify it as an optimization problem over policy preferences, constrained by preferences. If finding an optimal solution takes more than polynomial time, all the polling, deliberation, and goodwill in the world might still not be enough to arrive at a perfect (i.e. Nash, and/or Pareto-optimal) solution. In the meantime, politicians will make mistakes, politics will stay messy, and history will stay unpredictable.

Finally, a P!=NP proof puts us in an interesting place philosophically, especially when it comes to understanding intelligence. As with engineering, optimal solutions to many problems will always out of reach for artificial intelligence. Unless computation becomes much cheaper, a computer will never play the perfect game of chess, let alone the perfect game of go. Other difficult (and perhaps more useful) problems will also stay out of reach.

For those who take a materialistic view of human intelligence -- that is, your mind/consciousness operates within a brain and body composed of matter -- a P!=NP proof puts the same constraint on our intelligence. As fancy and impressive as our neural hardware is, it's bound by the laws of physics and therefore the laws of algorithmic computation. If that's true, then there are difficult problems that no amount of information, expertise, or intuition will ever solve.

The jury is still out on Deolalikar's paper. But if he's right, this isn't guesswork or extrapolation -- it's a matter of proof.

* Your friend solves logic puzzles; she's not an oracle. To give you the right answer, she needs to have all the relevant information when she begins, so it's no good asking her to pick winning horses in the Kentucky Derby, or guess how many fingers you're holding up behind your back.

** To be precise here, all P problems are also NP. If I can give you the solution in polynomial time, I can also verify the solution in polynomial time. When I say "truly NP," I mean problems that are NP but not P, such as NP-complete problems. This is the class of problems whose existence has been unclear.

Inkscape

If you haven't tried Inkscape yet, you should. It's an open-source graphics program, like Illustrator, but cheap. Very functional, totally free, the built-in documentation is pretty solid. Since it's open source with a good user base, I expect development to come along rapidly. Worth a look.
Here's a link to the Inkscape showcase.

Here's a link to an essential tool in Inkscape: converting bitmaps to vector graphics.

Here's me showing off the result of an hour playing around with bitmaps, fonts, and vector paths: a spoof on the Baby Einstein logo.

"Fail small and fast"

Here's an interesting interview with Google's Peter Norvig on Slate's The Wrong Stuff (found via Marginal Revolution). The topic is Google's approach to innovation, error, and failure. I really like the admonition to "fail small and fast."

Just to chip in my own two cents -- I've discovered that academic culture* is intolerant towards failure, errors, and mistakes in general. (You learn this fast when you have a lot of bad ideas.) We treat research like building a space shuttle, not looking for a decent restaurant. We do a great job critiquing ideas and tearing apart arguments, but we're often not very good about brainstorming or being willing to fail publicly. We're very reluctant to accept the null hypothesis.

Why? It's always struck me as really odd that an occupation that is ostensibly about new ideas, innovation, and pushing the status quo ends taking such a slow-moving, conservative approach. I'm still not sure why. Some working hypotheses:

• Like politicians, professors' careers live and die by reputation. We can't afford to fail publicly. Those who do lose credibility and are forgotten.
  
• Grad school selects for the risk-averse. Bigger bets and more daring would benefit the field, but those sorts of people channel themselves to jobs that pay more and don't take 5 extra years of schooling to
  
• Committee decisions favor the cautious. Decisions about funding, tenure, and publication are all made by committee, usually anonymously.
• The old guard kills innovation. If you made your name building up theory X, and some upstart comes along arguing that theory Y is better, it's only natural to resist. I don't believe the strong, cynical version of this hypothesis, but I've seen a number of examples of generational inertia.
• The publication scale makes big bets risky. It takes several years to publish a book. If you have to write one book to graduate and another to get tenure, you don't have a lot of time to write failed books. Smaller increments might allow for more failure and therefore more innovation.
  

*This may be limited to political science; I'm not as immersed in the norms of other disciplines.

Link mishmash

Four links worth a quick visit.

1. "Minimum parking requirements act like a fertility drug for cars." Interesting NYTimes editorial on the consequences of government-mandated parking. I'm interesting in getting the libertarian take on this one. Do minimum parking requirements distort the market for land use by overproducing parking lots and roads, or do they facilitate commerce by lower transaction costs? Hat tip: David Smith.

2. An infographic from Bloomberg Businessweek on negative buzz, doping, and Lance Armstrong's reputation. The interesting thing here is the source: automated buzz tracking services applying sentiment analysis to the blogosphere. This is a technology I plan to use in my dissertation.

3. The myth of a conservative corporate America - A nice infographic on corporate donations to politics. Good use of FEC data.

4. Some good resources for screen scraping

"Information is good"

Picking up a thread from the monkey cage (great blog--worth a look), Slate has an article by Washington Post reporter Anne Applebaum arguing that "WikiLeaks' data-dump reporting simply makes a case for the existence of the mainstream media."

The article is a quick read. I want to pick it apart, and see where it takes us.

Things Applebaum gets:

• Facts need interpretation to be useful.
• In this instance WikiLeaks provided lots of facts and very little interpretation.

Things Applebaum misses:

• "Mainstream media" is not the only institution that can provide useful interpretation, context, expertise, etc.
• Transparency (making data public) breaks up mainstream media's monopoly on interpretation by allowing others (e.g. bloggers) to tell different stories.
  
• Transparency allows savvy readers to come to their own conclusions, instead of relying on reporters to decide which facts are important. Most people won't bother to do this. But it's useful to have multiple perspectives.
  

My takeaways:

• Data dumps don't replace mainstream media -- but they do threaten its monopoly on interpretation. The slightly defensive tone of the article suggests that Applebaum agrees, whether or not she wants to admit it. Don't get me wrong. I'm not a Big Media hater, but I am a big fan of transparent reporting.
• On a more philosophical level, the difference between raw data and interpretation leads to some interesting ideas about "information." Raw data is not actionable. Is it still information? For most people, the dense, jargon-laden text of military reports don't mean anything. Reading them doesn't give you any traction to change your opinion or do anything differently -- they are "uninformative." A few experts (e.g. troops in Afghanistan, military experts) know how to unpack and sift through these reports. Apparently, what constitutes "information" is in the eye of the beholder.
  

Mutiny, information technology, and technocracy

Responding to David Brooks, on the recent upsurge in technocracy and its risks. (This post was originally part of a discussion thread with friends, but I got into it enough to decide to put it up here.)


I'm convinced technology is increasing the marginal truthfulness of many progressive claims. I don't believe it fundamentally changes the relationship among individuals, economies, government, and other social institutions.

The example that comes to mind is mutiny. Mutiny was a huge worry for captains and navies for hundreds of years, right up until the invention of the radio. See mutiny for interesting reading, and watch Mutiny on the Bounty and The Caine Mutiny (starring Humprey Bogart!) for good fictional accounts of the psychology and institutions that shape mutiny.

Radio was a game changer. Since the invention and adoption of radio, mutiny has been almost unheard of, especially among large naval powers. (The Vietnam-era SS_Columbia_Eagle_incident is the exception that proves the rule.) Shipboard radios tighten the link between the captain's authority and the worldwide chain of command. It makes escape extremely unlikely for mutineers. However, desertion, disobedience, cowardice, incompetence, corruption, theft, etc. are still problems for ships and navies.

As I see it, mutiny was already a marginal activity -- very risky for the mutineers, with a low probability of success -- and radio pushed the marginal success rate much lower. But mutiny is just one act. From the perspective of naval efficiency, radio changed the balance of power, but didn't fix the underlying social problems of enforcing discipline and coordinating action. Radio caused changes in the social structure of ships, but they didn't fundamentally alter the problems that navies face.

Information technology is doing a similar thing today. It lowers the cost of storing, transmitting, aggregating, and manipulating data. Where lower transaction costs can solve social problems, the progressives (and I'm one of them, cautiously) are right to be optimistic.

But many kinds of information have been cheap for a long time. Socially, we haven't solved the problems of greed, lying, bureaucratic turf wars, bullying, corruption, graft, incompetence... When *those* are the real causes, changes in information technology can't be expected to help nearly as much. We need to invent better institutions first.The example that comes to mind is mutiny. Mutiny was a huge worry for captains and navies for hundreds of years, right up until the invention of the radio. See mutiny for interesting reading, and watch Mutiny on the Bounty and The Caine Mutiny (starring Humprey Bogart!) for good fictional accounts of the psychology and institutions that shape mutiny.Radio was a game changer. Since the invention and adoption of radio, mutiny has been almost unheard of, especially among large naval powers. (The Vietnam-era SS_Columbia_Eagle_incident is the exception that proves the rule.) Shipboard radios tighten the link between the captain's authority and the worldwide chain of command. It makes escape extremely unlikely for mutineers. However, desertion, disobedience, cowardice, incompetence, corruption, theft, etc. are still problems for ships and navies.As I see it, mutiny was already a marginal activity -- very risky for the mutineers, with a low probability of success -- and radio pushed the marginal success rate much lower. But mutiny is just one act. From the perspective of naval efficiency, radio changed the balance of power, but didn't fix the underlying social problems of enforcing discipline and coordinating action. Radio caused changes in the social structure of ships, but they didn't fundamentally alter the problems that navies face.Information technology is doing a similar thing today. It lowers the cost of storing, transmitting, aggregating, and manipulating data. Where lower transaction costs can solve social problems, the progressives (and I'm one of them, cautiously) are right to be optimistic.But many kinds of information have been cheap for a long time. Socially, we haven't solved the problems of greed, lying, bureaucratic turf wars, bullying, corruption, graft, incompetence... When *those* are the real causes, changes in information technology can't be expected to help nearly as much. We need to invent better institutions first.">

Top secret America: Simultaneously sinister and incompetent

The Washington Post just put out an investigative report on "Top Secret America" -- federal departments and agencies with top secret security clearance.

Here's the trailer. Yes, a 1:47 Flash video trailer for a report put out by a newspaper. It has a Bourne Ultimatum feel to it, right down to the percussion-heavy, digital soundtrack and gritty urban imagery. Anybody want to talk about the blurring lines between news and entertainment?

WPost claims that a fourth, secret branch of government has opened up since 9-11. They make this new branch out to be simultaneously sinister and incompetent (like Vogons). Watch the trailer and see what they're hinting at.

Some counterclaims: government is still learning how to make use of recent (i.e. within the last 30 years) advances in information technology. (Take this article in the Economist on the alleged obsolescence of the census, and data.gov as examples).

• Since the technology is new, it makes sense that new agencies would develop to handle the load.
• Information technology is useful, and the government has a lot of data to process, so it makes sense that there would be a lot of people involved.
• Since business tends to be more nimble about technology adoption, it makes sense that a lot of the work would be outsourced to private firms, at least initially.
• Since the process is unfamiliar, it makes sense that there would be some inefficiency.
• Since we want the system to be robust to failure, it makes sense that there would be some redundancy.

I agree with the Post that "we want to get this right," but let's back off the hype and the alarmism a little bit.

a jaundiced formula for spinning educational research into something that sounds interesting

Here's a jaundiced formula for spinning educational research into something that sounds interesting. Most researchers and reporters seem to follow this formula pretty closely*.

1. Sample a bunch of kids in category A, and a bunch of kids in category B.

Ex: A kids have computers in the home; B kids don't
Ex: A kids are white; B kids are nonwhite
Ex: A kids go to charter schools; B kids don't

2. For each group, measure some dependent variable, Y, that we care about.

Ex: grades, SAT scores, dropout rates, college attendance, college completion, long term impacts on wages, quality of life, etc.

3. Compare Y means for group A and group B.
3a. If the means differ and the A versus B debate is contested, take a side with the group A.
3b. If the means don't differ and many people support one option, take the opposite stance. (Ex: "Charter schools don't outperform non-charter schools")
3c. If neither of those options works, continue on to step 4.

4. Introduce a demographic variable X, (probably gender or SES) as a control or interaction term in your regression analysis. It will probably be significant. Claim that A or B is "widening the racial achievement gap," or "narrowing the gender gap," etc. as appropriate.

Papers following this formula will frequently be publishable and newsworthy. (You can verify this, case by case, with the studies cited in that NYTimes article.) They will rarely make a substantive contribution to the science and policy of education. Awful. Awful. Awful.

Why? Because this approach is superficial. The scientific method is supposed to help us understand root causes, with an eye to making people better off. But that depends on starting with categorizations that are meaningfully tied to causal pathways. The distinctions we make have to matter.

In a great many educational studies, the categories used to split kids are cheap and easy to observe. Therefore, they make for easy studies and quick stereotypes. They feed political conflict about how to divide pies. But they don't matter in any deep, structural way.

Example: Does having a computer in the house makes a kid smarter or dumber? It depends on how the computer is used. If the computer is in the attic, wrapped in plastic, the effect of computer ownership on grades, SAT scores, or whatever will be pretty close to zero. If the computer is only used to play games, the effect probably won't be positive; and if games crowd out homework, the effect will be negative. No real surprises there. And that's about as far as these studies usually go. "Computers not a magic bullet. Next!"

This is more or less the state of knowledge with respect to school funding, busing, charter schools, etc. We know that one blunt policy intervention after another does not work miracles. We haven't really gotten under the hood of what makes the complex social system of education work. It's like coming up with a theory of how airplanes fly based on the colors they're painted. ("White airplanes travel slower than airplanes painted camouflage colors, but tail markings have little effect on air speed.) You may be able to explain more than nothing, but you certainly haven't grasped the forces that make the system work.

To say the same thing in different words, scientists are supposed to ask "why?" Studies that say "kids in group A are more Y than kids in group B" doesn't answer the why question. They are descriptive, not causal. Without a deeper causal understanding of why schools work or don't work, I don't think we're ever going to stop chasing fads and really make things better.


*This is an epistemological critique of just about every quantitative article on education. In general, I'm supportive of the increasing influence of economic/econometric analysis in education policy, but this is one area where we quants may be making things worse, not better. Hat tip to Matt for sending the article that reminded me how much the failings of this literature frustrate me.

Google code -- pros and cons?

Anybody have thoughts/experience on the pros and cons of Google code?

Reliability

I'm sold on the argument that Krippendorff's alpha is the way to go for most reliability calculation. Unfortunately, it's been hard to find code to calculate it. Here are some of the best links I've run across.

The NLTK package for python has code for computing alpha. It looks like this does basic nominal calculation; I don't know if/how it copes with missing data.

The concord package in R does nominal, ordinal, interval, and ratio versions of alpha. It looks like this might not be maintained anymore, but it works.

Here's a nice page of resources by computational linguists Artstein and Poesio. Unfortunately, what they show is mainly that there aren't very good resources out there. Their review article is very good -- the best treatment of reliability I've seen in the NLP community so far.

Deen Freelon has some links to reliability calculators and resources, including two nice online reliability calculators: Recal-OIR, and Recal-3.

Krippendorff's oddly-formatted, information-sparse web page. He invents the best measure for calculating reliability, then keeps a lid on it. Less animated bowtie dogs, and more software, please!

Matthew Lombard has a nice page on reliability statistics and the importance of reliability in content analysis in general.

Beg: Does anybody know how to compute K's alpha for a single coder?

I have data coded by several coders and need to know who's doing a good job, reliability-wise, and who's not. At a pinch, I'd be willing to use a different reliability statistic, or even an IRT model. It just needs to be statistically defensible and reasonably easy to code.

How to catch a dirty pollster

Just a couple hours ago, Markos Moulitsas (you know, the Daily Kos) announced that he's suing his former polling group, Research 2000. Evidently, they faked a lot of their data in the last 19 months, and didn't even have the decency to fake it well.

The statistical case against R2000 is made here. Nate Silver chimes in here.

The punchline:

"We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls."

What's interesting to me about the situation is how clear-cut the evidence is. Mark Grebner, Michael Weissman, and Jonathan Weissman (GWW), conduct three tests to show that the numbers are cooked. Each statistical test gives results that are devastatingly improbable.

• Probability of odd-even pairs between men and women: 776 heads on 778 tosses of a fair coin, or about 1 in 10²³¹.
• Probability of consistency of results in small samples: ~1 in 10¹⁶.
• Probability of missing zeroes in a time trend: ~1 in 1016.

Statistically, there's almost no wiggle room here. Until the R2000 folks come forward with some really compelling counterevidence, I'm convinced they're frauds.

An interesting feature of the last two patterns is that they hint at overeagerness to give results. Patterns in the data are *too* clear, and the detailed crosstabs supplied by R2000 made it easier to catch them. If they had been willing to dump some random noise into the equations -- blurring the results -- it would have been much harder to build a conclusive case against them.

This jives with my experience doing survey research for clients: nobody wants to pay for null results. Even when your data collection is legitimate, there's always pressure to twist and squeeze the data to wring out "findings" that aren't really there. Three cheers for transparency and the scientific method.

Reasons to study political blogging

I'm working like crazy on my dissertation prospectus. Data work, lit reviews, etc. To escape from early research purgatory, I plan to blog parts of the prospectus as I write them.

I'll kickoff today with introductory definitions and motivation. Feedback is much appreciated. Beware of dry, academic writing!

What is a blog?
Paraphrasing wikipedia, a blog is a website containing regular entries ("posts") of commentary, links, or other material such as photos or video. On most blogs, posts are displayed in reverse-chronological order -- the most recent post appears first. Although most blogs are maintained by individuals, some are run by small groups, and blogs speaking on behalf of corporations, churches, newspapers, political campaigns, etc. are increasingly common. Many blogs focus on a specific topic, ranging from broad to narrow: entertainment, cooking, astronomy,the Detroit Tigers, to cold fusion. For my dissertation, I plan to focus on political blogs.

Why study political blogs?
Here are five reasons to study political blogs.

1. Blogs are public facing. Lots of people read them, including politicians and journalists. The extent to which blogs are replacing mainstream media is an open question, but it's certain that blogs have come to play an important role in public discourse, with real impact on politics.
2. Bloggers span a wide variety of opinions. The blogosphere embraces everyone from conservative wingnuts to liberal moonbats to political moderates. Some political bloggers are politically omnivorous, writing about anything political. Others focus on specific issues and topics: foreign policy, Congress, feminism, etc.
3. Bloggers include both experts and amateurs. Dividing the same pie in a different direction, many A-list bloggers (e.g. Andrew Sullivan, Ariana Huffington, Glenn Reynolds, Michelle Malkin) clearly qualify as political elites: they are experts, immersed in politics, well-informed and well-connected. Other political bloggers are more obscure, casual -- closer to the average Joes who make up the "mass public."
   
4. Blogs are updated frequently. This has two nice consequences. First, frequent posts allow us to replay bloggers' reactions to events as they unfold. Second, frequent posts mean we have a lot of posts to work with.
   
5. Blogs are archived publicly. Unlike most forms of political speech and action, blogging leaves a permanent data trail.
   

The combination of these attributes creates a kind of perfect storm for social science. Understanding the flow of opinions and ideas has always been difficult for social scientists, because most of our data have come from surveys.

AI for Jeopardy: IBM's Watson

In case you haven't heard, IBM is creating a supercomputer to play Jeopardy. The idea is to build AI that can answer factual questions in regular, spoken English. The system, named "Watson" after two longtime IBM presidents, has been in development for a couple years now, and is starting to beat strong-ish human competitors. It's slated to play against past Jeopardy grandmasters sometime this fall.

Here's a short promotional video by IBM.

Here's an extended article discussing the challenge and the technology in NYTimes.

Related: from xkcd.

Scientific revolutions ~ Disruptive technologies

These two wikipedia pages are really interesting to read side by side: The Structure of Scientific Revolutions and Disruptive Technology. The first is Thomas Kuhn's 50-year-old-but-still-revolutionary theory of how scientific theories advance. The other is Clayton Christensen's more recent theory of why many businesses fail to respond to innovation. Despite the fact that Kuhn was a physicist and scientific historian and Christensen is a business school professor, the two have a lot in common.



Here's a list of important concepts from both theories. I've paired roughly equivalent concepts here. I'll let you look them up from wikipedia on your own.

Kuhn / Christensen
Scientific revolutions ~ Market disruption
Paradigm ~ Value network
Paradigm shift ~ Market disruption
Coherence ~ Corporate culture
Normal science ~ Sustaining innovation
Anomalies ~ New-market disruption
New theory ~ Disruptive innovation
??? ~ Low-end disruption
??? ~ Up-market/down-market
Incommensurability ~ ???
Pre-paradigm phase ~ ???
Normal phase ~ Market growth
Revolutionary phase ~ Market disruption

One thing that strikes me as potentially interesting is the places where the two theories do *not* overlap. As a businessperson, Christensen is more interested in the development of markets and the flow of revenue. Kuhn is more interested in the change in theories over time. It strikes me that each approach may have something to offer the other.

* In science and academia, what does it mean to be "down-market?" Which departments today are incubating the revolutionary theories of tomorrow?
* What does the idea of incommensurability imply for business practice? Anecdotally, disruptive companies often have business models and cultures that are dramatically different from established companies. Should that change the way we think about entrepreneurship and venture capital?

Zinger or below the belt? WSJ argues that "liberals and Democrats" are economically unenlightened

An editorial in today's WSJ (Are You Smarter Than a Fifth Grader? ) argues that "self-identified liberals and Democrats do badly on questions of basic economics."

Written by a George Mason economist, the article is admirably transparent in its reasoning. The analysis turns on a battery of Econ 101-style questions on a Zogby poll (e.g. "Restrictions on housing development make housing less affordable.) It turns out that self-identified Republicans and libertarians score substantially better on this quiz than Democrats. Conclusion: the left doesn't understand, or is unwilling to accept, the fundamental economic tradeoffs that exist in any society. In the author's words, the left is "economically unenlightened."

Usually when I see this kind of thing in the WSJ, I'm inclined to ignore it as partisan sniping. In this case, they lay out their methodology thoroughly enough to invite inspection. And against my will, I find myself agreeing, because I don't see anything wrong with the analysis. Here's my reasoning.

The first thing to check is the quality of evidence. In order to score respondents' answers, the researchers had to designate right and wrong answers to each question on the quiz. What questions, exactly, were asked? Did the scoring reflect objective truth, or was there libertarian dogma in the way things were framed?

Here are the 8 questions.

Question	"Unenlightened" Answer	Validity
1. Restrictions on housing development make housing less affordable	Disagree	High
2. Mandatory licensing of professional services increases the prices of those services	Disagree	High
3. Rent control leads to housing shortages	Disagree	High
4. Free trade leads to unemployment	Agree	Low
5. Minimum wage laws raise unemployment	Disagree	Medium
6. Overall, the standard of living is higher today than it was 30 years ago	Disagree	High
7. A company with the largest market share is a monopoly	Agree	Medium
8. Third World workers working for American companies overseas are being exploited	Agree	Low

On the whole, these questions strike me as having high validity. They measure what they intend to measure. A respondent who answers these questions correctly probably does have a better understanding of the likely consequences of economic policies. And therefore, it looks like a substantial part of the left's constituency is unwilling to come to grips with hard choices.

I'm not sure I want to believe that. Does anybody see a way out of this conclusion?



Notes on specific questions:
Questions 1 through 5 focus on fundamental tradeoffs in price, quantity supplied, and market intervention. The first three are well-grounded in evidence. The fourth is true -- in the long run. The last one is contested, but (having read up on the subject for a final debate in a policy analysis class) the proponderance of evidence supports this conclusion. Bottom line: both theory and evidence strongly suggest that the tradeoffs described in these questions are real forces in society.

Question six is a simple factual question about recent economic history. Question seven is a vocab question.

Question eight is more values-based. A typical economist will tell you that Third World workers aren't being exploited, because they voluntarily choose to accept and continue in those jobs. Companies aren't exploiting people; they're giving them new opportunities. The counterargument is that (some) workers are led into sweatshop jobs under false pretenses, and held there against their wills. This is exploitation. Additionally, one could argue that it is "exploitation" in a moral sense for a company to pays its workers only Third World wages plus a fraction when it could pay more.