The Lowly Wonk

Moving to a new home...

2011-10-24T10:38:00.000-07:00

I've been blogging at http://lowlywonk.blogspot.com for most of my time in grad school, and now it's time for a change. I'm still interested in public policy, but my so much of work is computational now. It just makes more sense to have "computational" in the name of the blog.

So please point your blog readers to http://compSocSci.blogspot.com so the adventure can continue.

(The contents of this blog will remain online, searchable, etc. for the forseeable future.)

Latex: tikz

2011-09-07T05:52:00.000-07:00

In the runup to APSA, I discovered the absolutely fantastic tikz package for diagrams in LaTex.

See the gallery of example here.

Here are some links that were helpful for preparing my presentation:

Topics that came up at ASPA, part 1.

2011-09-06T12:43:00.000-07:00

I'm looking over my notes from APSA, highlighting papers, ideas, etc. that looked worth passing along. As usual, some of these were new to me, and others are just worth knowing about.

From a section on game theory and experimentation:

More to come soon...

Slides from APSA: Why do people blog about politics

2011-09-05T10:52:00.001-07:00

Back from APSA!

Here is the final version of my presentation slides. I'm not going to post the paper because it's in the process of splitting into three, but I'm happy to share by email if you're interested.

On with the job search!

Hard-learned lessons on data merging

2011-08-30T10:26:00.000-07:00

For my APSA paper this year, I had to merge two survey data sets, and do a bunch of work with imputation and instrumental variables. To complicate things, I was working with a student on the merge.

Here are some lessons learned:

1. Agree on an outline before doing the data work.
Know the main steps going in, and write them as comments in the code. This will make the logic of the data work more clear, which is helpful for debugging. For long merges, it can be motivating to see your progress as you go along. Also, if the steps need to change, it's good to realize why the original plan didn't work.

2. Stick to an intuitive naming convention for variables, data sets, etc.
Yes, you'll have to type out longer variable names, but you'll save a ton of time in debugging. For instance, "october_cleaned_dataset" instead of "d2.10c".

3. Store files as .csv, not .dat.
R's "save" command isn't fully compatible across versions and operating systems. Instead, save small-to-moderate datasets as .csv files. That way, compatibility won't be an issue. (Plus you can open the same files in other programs, like STATA or excel.)

4. Use names, not numbers.
Don't use commands based on row or column numbers: data_set[,c(1:4,7,8,23)]. They're hard to read, and they're brittle: if a new column gets inserted in data_set, that code isn't going to work anymore. Instead, use variable names: data_set[,c("time1","time2","time3","time4","age","gender","year")]. Regular expressions can be very helpful here: data_set[,append(grep("time.", names(data_set)), c("age","gender","year"))]. Yes, it takes longer to write, but you'll save a huge amount of time debugging.

The world's fastest demo for RStudio in EC2

2011-08-08T11:05:00.000-07:00

Last week I posted instructions to get started with RStudio on Amazon's Elastic Compute Cloud. Here are new and improved* instructions (docx, pdf). These steps should be enough to get you into the cloud in 15 minutes or less, for free.

Please let me know in the comments if you have any trouble or questions with this demo. I'm trying to lower the startup costs for people to do computational social science, so I'm happy to be a resource for others working their way down the cloud computing path.

Cheers!

PS - The instance comes with several fun R libraries pre-installed: tm, igraph, and twitteR.

*I've 1) dropped several steps that aren't necessary for running RStudio, 2) added a few screenshots, and 3) clarified a steps that were giving people trouble. Thanks again to Kevin J. for putting together the original slides.

Mining and visualizing twitter from RStudio in EC2

2011-08-01T11:23:00.000-07:00

Here's code I'm going to use for my ICPSR class today. This is set up to run immediately from an EC2 instance of my AMI agongRStudio2 (ID:ami-1bb47272). It's the simplest introduction to text mining I've been able to pull together so far.

Step-by-step instructions for getting started in EC2 are here (pdf and docx). These are intended to get you started in command-line R. For this exercise, we want to use the RStudio GUI instead, so there are a few changes.

1. On step 6, use this Community AMI: agongRStudio2 / ami-1bb47272

2. On step 8, you don't need to download the keypair. "Proceed without a keypair" instead.

2. On step 9, you also need to enable port 8787, the port the RStudio server uses.

3. On step 11 stop following the tutorial. Instead, open up your EC2 URL in your browser, with port 8787. It will look something like this:

http://ec2-123-45-67-890.compute-1.amazonaws.com:8787/

4. I'll give out the username and password in class. If you're not in the class, email me and I can clue you in.

5. Here's a first script to run

library(twitteR)
library(tm)
library(wordcloud)

#Grab the 200 most recent tweets about #bachmann
#http://www.slideshare.net/jeffreybreen/r-by-example-mining-twitter-for
k = 200
my_tweets <- searchTwitter("#bachmann", n=k)

#Convert tweet status objects to text
#http://www.r-bloggers.com/word-cloud-in-r/
my_text <- data.frame( text=unlist( lapply( c(1:k), function(x){my_tweets[[x]]$text} ) ) )

#Convert text to a tm corpus object
my_corpus <- Corpus( DataframeSource( my_text ) )
my_corpus <- tm_map(my_corpus, removePunctuation)
my_corpus <- tm_map(my_corpus, tolower)
my_corpus <- tm_map(my_corpus, function(x) removeWords(x, stopwords("english")))

#Convert corpus to matrix
tdm <- TermDocumentMatrix(my_corpus)#, control = list(weighting = weightTfIdf))
m <- as.matrix(tdm)

#Get features and frequencies
v <- sort(rowSums(m),decreasing=TRUE)
d <- data.frame(word = names(v),freq=v)

#Display as a word cloud
wordcloud(d$word,d$freq,min.freq=5,use.r.layout=T,vfont=c("sans serif","plain"))

#Basic bluster analysis of words
#From: http://www.statmethods.net/advstats/cluster.html
m2 <- m[colSums(m)>15,]

dist_matrix <- dist(m2, method = "euclidean") # distance matrix
fit <- hclust(dist_matrix, method="ward")
plot(fit) # display dendogram

We're going to be trying this in class. I have 20 minutes budgeted, so hopefully it's really this easy.

PS - Don't forget to terminate your EC2 instance when you're done, or you will use up your free hours, then run up a smallish (~50 cents/day) Amazon bill until you remember

Brand sentiment showdown...

2011-07-25T19:50:00.000-07:00

From FlowingData. I may use this in my ICPSR class on computational social science next week. What do you think?

Brand sentiment showdown: "

There are many brands on Twitter that exist to uphold an image of the company they represent. As consumers, we can communicate with these accounts, voicing praise or displeasure (usually the latter). Using a simple sentiment classifier¹, I scored feelings towards major brands from 0 (horrible) to 100 (excellent) once a day for five days.

The above for example, shows scores for Netflix, Hulu, and Redbox. Netflix had the lowest scores, whereas Redbox had the highest. I suspect Netflix started low with people still upset over the price hike, but it got better the next couple of days. Then on Saturday, there was a score drop, which I'm guessing was from their downtime for most of Saturday. Hulu and Redbox, on the other hand, held more steady scores.

As for auto brands, Toyota clearly had the lowest scores. However, Lexus, which is actually a luxury vehicle division of Toyota had the highest scores in the high 90s to 100.

How about the major mobile phone companies, AT&T, Verizon, and Sprint? Verizon scored better initially, but had lower scores during the weekend. Not sure what was going on with Sprint.

Between Twitter and Facebook, there was obviously some bias, but Twitter faired slightly better. Twitter scored lower than I expected, but it probably has to do with bug reports directed towards @twitter.

Is Domino's Pizza good now? Papa John's stayed fairly steady while Pizza Hut scores were sub-par.

Finally, as a sanity check, I compared airlines like Breen did in his tutorial. Results were similar with JetBlue and Southwest clearly in the positive and the others picking up the rear.

Any of these scores seem surprising to you?

Jeffrey Breen provides an easy-to-follow tutorial on Twitter sentiment in R. The scoring system is pretty basic. All you do is load tweets with a given search phrase, and then find all the 'good' words and 'bad' words. Good words give +1, and bad words give -1. Then a tweet is classified good or bad based on the total. Then to get a final score, only tweets with total of +2 or more or -2 or less are counted. The final score is computed by dividing number of negative tweets divided by total number of 'extreme' tweets. Obviously this won't pick up on sarcasm, but the scoring seems to still do a decent job. I wouldn't make any important business decisions based on these results though.

The new FlowingData book is available now.

Automated snowball census

2011-07-15T17:12:00.001-07:00

After a lot of work, I pushed the new version of the paper to SSRN today. Getting ready to pack it up, send it off for review, and move on to the next thing.

How are new media reshaping politics? Take 2

2011-07-12T09:53:00.000-07:00

Yesterday I posted on Hidman's "missing middle" hypothesis. Kudos to my good friend Ben Peters for some great, thought-provoking responses. Today, I'm going to push forward and respond to another great thinker in this area: Benkler's theory of the "networked public sphere."

Benkler and the Networked Public Sphere

On the other side of the debate, Yochai Benkler is an Internet optimist. He argues that many-to-many communication will invigorate the public sphere, leading to broader intake of ideas, better discussion, and ultimately better governance. Benkler is very critical of the media oligopoly of the mid-20th century, which he says was heavily influenced by money and ideology, and excersized outsized control on public access to information. According to his account, the current proliferation of online information sources is certainly better than being dependent on a handful of corporate broadcasters, even if it still falls short of utopia.

This picture of the public sphere is appealing and not entirely untrue. I want to believe it. However, Benkler fails to take into important and well-established facts about American political system.

First, most citizens in the U.S. are poorly equipped to deal with political information. Converse's half-century-old finding that as many as 90 percent of Americans are "innocent of ideology" (i.e. they have no idea what "liberal" and "conservative" mean) has been replicated and extended many times. Most voters don't know how government works, they don't know how it's supposed to work, and they don't care to find out. True, partisan cues, endorsements, and heuristics can sometimes bring voters up to speed enough to fill out a ballot, but these heuristic strategies cannot inform most citizens for participation in the public sphere the way Benkler imagines. We must distinguish between the handful of citizens who are motivated and equipped to reason about politics, and the majority who are not so prepared or inclined. Benkler's optimism really only extends as far as the electorate is capable of reasoning about democracy.

Second, Benkler ignores the structure of government and policymaking. He treats "government" as a unitary actor, and makes only passing reference to elections and political parties. Benkler is painting with a broad brush, so perhaps he can be forgiven for ignoring the institutional details of representation and government in American politics. However, those details are likely to matter, deeply.

Consider: primary responsibility for lawmaking in the U.S. falls to elected legislators. These legislators are influenced not only by the ebb and flow of ideas in public debate, but by their ability win in zero-sum, partisan elections. Proliferation of information sources may affects public debate for the better, but it also affects the electoral pressures faced by public officials. We have strong reason to believe that access to additional channels, selective exposure, and ideological pandering are leading to increased polarization in the electorate. What if this polarizing electoral effect dominates the enriching discursive effect that Benkler outlines?

I'm sympathetic to the the idea of a networked public sphere. As I said earlier, I really want it to be true. But Benkler's picture ignores key institutions in American politics, like elections and parties, so I have a hard time placing much faith in his predictions. We need to think carefully about the interplay of partisanship, ignorance, and representative government with technologies that allow cheap, many-to-many communication.

How are new media reshaping politics? Take 1

2011-07-12T09:35:00.000-07:00

I've been studying political blogging for a couple years now, and I'm getting ready to bring it all together into a dissertation. That means it's time to move past statistics and data, and start thinking in terms of Big Ideas.

As I see it, the pressing question is "How are new media (including blogs) reshaping American politics?" This is a big question -- one that certainly matters outside of academia. But that won't stop me from writing about it in a dry, academic way. :) To my mind, Matthew Hindman, Yochai Benkler, and Cass Sunstein have put forward the three leading, competing theories for answering this question. This week, I'm going to make a first attempt at responding to and synthesizing their ideas.

Feedback and constructive criticism are very welcome.

Hindman and the Missing Middle

Matthew Hindman is an Internet pessimist. In his book, The Myth of Digital Democracy, he argues that the web has exacerbated the "rich get richer" tendencies of media markets, leading to greater inequality. To back up his assertion, he shows that links and traffic to web pages follows a power law distribution. He also interviews top 40 bloggers and claims that they are overwhelmingly white, male, high-income, and educated. His analysis suggests that the people with big audiences online are no different from those offline. Hindman labels this dramatic inequality between popular and unpopular sites "the missing middle."

However, Hindman's line of attack has two important weaknesses. First, he has no counterfactual. The distribution of online audiences is dramatically unequal, but the same is (and was) probably true offline as well. Certainly, Barack Obama, Michelle Bachman, and Thomas Friedman have daily audiences that are orders of magnitude larger than mine or yours. The same was true of their counterparts before the Internet. Audiences online are distributed unequally, but are they more unequal than those that existed offline, before the Internet? Hindman does not answer this question, and I suspect the answer is no.

Second, Hindman ignores the potential for indirect influence. The Drudge Report is one of the most heavily trafficked blogs* on the Web, but Drudge himself writes almost no content. Instead, the site features links to stories elsewhere on the Internet. How then do we think about Drudge's influence? He inserts no new ideas into public debate, but exercises some ability to influence which ideas get attention. By linking to other authors' stories, Drudge allows those authors to exercise indirect influence on his readers.

Drudge is an extreme case of the common online practice of linking. Linked content intrinsically gives others indirect influence. It is not unique to the online world (think of citations, endorsements, recommendations), but it is probably more common there. Network theory shows us that all else equal, more re-linking leads to more egalitarian distribution of indirect influence. By focusing only on direct readership, Hindman misses this possibility.

The bottom line: Hindman is the skeptic in this debate, arguing that the Internet means business as usual for participation, voice, and influence. He's only right as long as we assume that 1) offline participation is not also unequal, and 2) only direct influence (i.e. readership and web traffic) matters.

+Computation: Got an AWS in Education grant!

2011-07-11T10:35:00.000-07:00

I just received a generous grant for usage on Amazon's Web Services -- cloud computing, storage space, and bandwidth. This is just in time for a bunch of heavy-duty text crunching I've been planning to do. Thank you, Amazon!

Political science and big data resources from JITP

2011-06-22T06:42:00.000-07:00

I'm cleaning out papers from my files, tired of carrying around all these dead trees. Here are notes on nifty resources mentioned at the JITP conference a few weeks ago.

TDT: topic detection and tracking (http://projects.ldc.upenn.edu/TDT/)

Socrata, the Open data company (http://www.socrata.com/)

Google's Data Liberation Front (http://www.dataliberation.org/)

TESS: Time sharing experiments in the social sciences (http://www.tessexperiments.org/)

TREC (Text retrieval conference) benchmark data sets (http://trec.nist.gov/data.html)

And the good old American National Election Study (ANES) (http://www.electionstudies.org/)

Laptop vs Clipboard

2011-06-20T08:28:00.000-07:00

Things that could destroy my:

Laptop	Both	Clipboard
Coffee Falling Power surges Worms and viruses Rain Babies EMPs Losing the power cable Tripping on the power cable Planned obsolescence Stomping Yogurt Crumbs The AllSpark Big magnets	Fire Karate experts Garbage compactors Lightsabers	Termites Beavers Very strong wind

PS: This list was created as my flight into Detroit was descending, and "all personal electronic items" had to be switched off.

Resources from PolNets

2011-06-18T08:19:00.000-07:00

Note from Saturday, at the Political Networks conference. The presentation went swimmingly. I think there are more sociologists than political scientists here today.

Here are links to nifty resources referenced in talks:

Affective Norms for English Words (ANEW): http://csea.phhp.ufl.edu/Media.html#bottommedia

Networks, Computation, and Social Dynamics Lab at UC Irvine: http://www.ncasd.org/public_html/

OpenCongress RSS feeds: http://www.opencongress.org/about/rss

Public.Resource.Org: https://public.resource.org/

Netroots Nation

2011-06-16T22:05:00.000-07:00

I'm at the Netroots Nation conference in Minneapolis, really enjoying it. Some reactions from day one:

* Lots of discussion of messaging and issue ownership. This is an area that academic researchers haven't really embraced, especially us quants. We should.

* The community is really open and egalitarian. I sat in the same panel with Markos Moulitsas for half an hour without realizing it.

* I've been impressed by the sophistication of research design in analytics and message experiments. It's got me thinking about what's the next step after AB testing... There's room for innovation here.

* The left-meets-right happy hour was kind of a flop. Where were all the Republicans?

Slides from JITP: The Future of Computational Social Science

2011-05-17T07:06:00.000-07:00

I'm at JITP's conference on the future of computational social science this week. Really interesting gathering of social scientists (and a smattering of CS people) interested in computational social science. I'll blog more thoughts on the conference once it's over (tonight).

For now, I'll post the resource I already have put together: my conference slides.

An Automated Snowball Census of the Political Web - JITP 2011

View more presentations from Abe Gong

Computational social scientists: a draft directory and basic survey results

2011-05-12T05:58:00.000-07:00

Last week, some of us* at Michigan's Center for Complex Systems circulated a survey of computational social scientists -- trying to find out who self-identifies as a compSocSci person and what they study, so that they can be in touch with each other.

We had just under 100 responses, from people at many different institutions, working in a wide variety of areas. Here are some early results.

First, the obligatory word cloud. This isn't particularly scientific, but it illustrates the concepts that people find important in this space. Not surprisingly, we had a strong showing from network people and agent-based modelers.

We also asked about broad areas where people had formal training and were currently working. The two are pretty similar, so I'll just show the graph on training.

More results, and a revised version of the directory will be forthcoming in a couple weeks. Please let us know if you have any questions. We hope these will be useful resources for the community.

Click here to take the survey. We'll keep it open for another couple weeks, so that responses can continue to trickle in.

Click here for the directory in pdf format. (To avoid spam, this doesn't include email addresses. Email me if you want a copy that includes emails.)

* Scott Page, Dan Katz, and I

R code to remove the second line of a Qualtrics .csv

2011-05-11T12:16:00.000-07:00

I love Qualtrics, but its data export does this obnoxious thing. Instead of exporting a regular .csv file, it exports a csv with two header rows. The first one contains short variable names (e.g. Q1, Q2.2, Q3_TEXT) and the second one contains labels ("How old are you?", "What is your email address?")

I keep having to figure out how to tell R how to deal with this messiness. It's not complicated, but I have to look up the read.csv documentation every time.

No more. Here's my code:

DF <- read.csv("my_file.csv", skip=2, header=F)
DF2 <- read.csv( "my_file.csv" )
names(DF) <- names(DF2)

I'm number one on Amazon turk!

2011-05-10T11:20:00.001-07:00

I just maxed out my credit card to get a whole bunch of work done on mTurk. For the moment, I'm the number one requester on the site!

Screen shot:

Early results look pretty good. A few turkers cheat, but I've been pleased with the good-faith effort most people seem to put into their work.

PS - I'm curious about what Randolph Stevenson (in the number 3 slot) is doing...

Working paper: An automated snowball census of the political web

2011-05-05T07:43:00.000-07:00

Here's my paper for the JITP Future of Computational Social Science conference in a couple weeks. This paper describes my process for using SnowCrawl and a highly trained text classifier to search out political web sites -- pretty much all of them -- on the web.

Final census results are available here. I'm planning to run another iteration of this census before too long. I welcome comments and suggestions.

The NYTimes is reporting Bin Laden dead

2011-05-01T20:00:00.000-07:00

Story here.

UPDATE 11:13: Whitehouse stream here:

JOIN THE LIVE CHAT

VISIT WHITEHOUSE.GOV

Computational politics: U.S. House legislation may move to XML?

2011-04-29T13:39:00.000-07:00

http://thehill.com/blogs/hillicon-valley/technology/158339-boehner-cantor-want-house-to-use-open-data-formats

This would be a huge boon to computational social scientists and groups like the Sunlight Project

The economics and politics of the Death Star

2011-04-29T04:42:00.000-07:00

The economics and politics of the Death Star

"What’s the economic calculus behind the Empire’s tactic of A) building a Death Star, B) intimidating planets into submission with the threat of destruction, and C) actually carrying through with said destruction if the planet doesn’t comply?"

Fun discussion with lots of analogies to history and current politics

Is bitcoin a decentralizing, democratizing agent?

2011-04-26T10:54:00.000-07:00

A response to Duane on the importance of bitcoin as a new, decentralized currency. (This has been a fun running debate among friends over the last couple weeks.)

I read his recent post as claiming "decentralized is better." My response: "sometimes, but not always." Here's my reasoning.

Point 1: Money is a figment of our shared imagination. It has value because we all collectively accept that it has value. There's nothing particularly special about green paper, or gold rocks, or any given set of bits, except for the common knowledge that other people will also accept those currencies in exchange for goods and services. In that sense, the valuation of any currency is already "democratic" -- its real value exists in the minds of a distributed network of people. People were using rare stones as currency long before governments got involved.

Point 2: Enlightened self-interest leads us to centralize some of the responsiblity for maintaining currencies. One of the main threats to any currency is counterfeiting, so it makes sense to centralize responsibility for preventing counterfeiting to mints and enforcement agencies like the secret service. Another threat is inflation (or deflation), which is why every modern state with a large economy uses a central bank to manage inflation via the money supply. A third "threat" is transaction costs, which is why we use credit cards for so many things -- the artificial "currencies" maintained by these corporations are so convenient that they have displaced state-authorized currency in many transactions*.

Point 3: As a side-effect of centralization, governments gain some power to regulate other uses of money. (Note that governments, like currencies, are also a figment of collective imagination. The Constitution is law because we all agree that it's the law.) Thus, the imposition of taxes, tariffs, and embargoes; and continuing efforts by organized criminals to circumvent the system by laundering money. These things are not necessarily good or bad. They depend on whether we approve of the use of government power in those areas. By and large, I imagine most people approve of governments shutting down mobsters and sex trafficking rings. Using the same suite of tools for defense spending, planned parenthood, agricultural subsidies, "bailouts," etc. is more controversial.

On the whole, I'd argue that people are better off because governments have these capabilities at their disposal -- especially people living in places where government is reasonably transparent and accountable to its citizens. We're better off because we've decided to pay taxes for roadways, police stations, and schools. We're better off because the value of a dollar is reasonably stable. We're better off because corporations (especially publically traded firms) are forced to keep a strict accounting of their transactions. We're better off because the FBI and CIA can use financial information to crack down on terrorists and organized criminals.

Putting that all together, I see bitcoin as an attempt to float an unregulateable currency using P2P technology and cryptography. By construction, such a currency would make it very difficult for government to intervene in the ways discussed in points two and three. Although I don't agree with everything our government does**, I don't see compelling reasons to deny ourselves the ability to use those tools as a society. Together, we're made better off by careful use of centralized financial regulation and lawmaking. That being the case, bitcoin seems partly radical, partly old hat, and partly a step backwards.

*Semantic question: are credit cards a centralized or decentralized currency? What about frequent flyer miles? What about Subway sandwich discount punch cards? At some level, the labels "centralized" and "decentralized" are too blunt to be really useful. This seems related to Duane's comments about needing both more and less regulation.

**Nobody does. That's the nature of compromise.

The best $5 I've spent all year

2011-04-25T11:39:00.000-07:00

I finally started using Amazon's EC2 yesterday. I've been meaning to learn it forever, but assumed it would be time-consuming to get registered, set up an instance, and so on.

Not true. Thursday morning at 10am, I registered for EC2. By 10:30 I had an instance of Drew Conway's Py/R AMI up and running, with several additional libraries installed, and a few GB of data I wanted to crunch uploaded to the server. Very fast turnaround.

Eight hours and $4.77 later, I'd crunched a lot of numbers -- by far my most productive workflow all week. Highly recommend it.

Scalable education

2011-04-22T10:48:00.000-07:00

I got a lot of great responses on my previous post about scalable education, and wanted to share them back out here. I also got a lot of questions on what I meant by "scalable" education. Let me speak to that first.

Thesis: the number of students getting a good education today is about 20 times the number of good teachers. Under our current classroom model, "number of high-quality teachers" is the limiting variable. By scalable education, I mean models of teaching that could feasibly grow to orders of magnitude greater than 20 -- systems that would allow one good teacher (plus the right support system) to teach 2,000 or 2,000,000 kids.

Here's a loosely annotated list of links to proposed models of "scalable education," sorted from least to most:

Jump - New curriculum deployment. Not bad, but not revolutionary.

PBS documentary on "digital learning" - This is a mixed bag. Some models (e.g. the Smithsonian scavenger hunt) are neat, engaging students in new ways. I was less impressed with several segments that are just trading classrooms for classrooms plus computers.

Teach for America - TFA's teachers do a lot of good, but they also tend move on quickly. (This may be changing.) On the other hand, TFA is very proactive about maintaining their alumni network. If this is going the be revolutionary, it will be as a policymaking network, more than a teaching force.

itunes-U - Possibly revolutionary. But content delivery < education.

TED - "The first major educational brand to emerge in a century." Very cool topics, but see previous.

Harvard's Distance Education - Lectures are free; course credit with the Harvard brand cost $. Many universities are doing this; Harvard is probably the best known. This is changing things, but I'm not sure how competition is going to play out in this space.

The Khan Academy - My number one vote for a potentially disruptive model of education.

Angry birds and agent-based modeling

2011-04-21T13:46:00.000-07:00

"How +1 modeling can strengthen the logic of simulation and keep your pigs intact."

What I've been watching lately

2011-04-20T12:09:00.000-07:00

Ralph Lagner's fascinating TED talk on reverse-engineering the Stuxnet worm. This is the best presentation on cyber-terrorism/security I've seen so far.

The bubble sort algorithm, illustrated as a Hungarian line dance. (via flowingdata) Fun, but a little long. What sort of dance could illustrate a heap sort?

A quick (4min) TED talk on Google's driverless cars. Nifty.

Code: a random web crawler

2011-04-19T06:35:00.000-07:00

This code crawls the web to generate a pseudo-random sample of web sites. Not my prettiest code, but it works and may save somebody an afternoon of coding.

random_crawler.tar.gz (14MB, zipped)

This script explores the web for a pseudo-random sample of sites. The crawl proceeds in a series of 100 (by default) waves. In each wave, 2,000 (by default) crawlers attempt to download pages. When page downloads are successful, all the unique outbound hyperlinks are stored to a master list. Then sites for the next round are sampled.

Sites from common domain names are undersampled, to avoid getting stuck (e.g. within specific social networking sites). When sites are selected for the next round, weights are equal x^(3/4), where x is the number of sites in the same domain.

After several waves, the sample should be well-mixed -- a pseudo-random sample of the web.

Note: This code is kind of sloppy, does not scale well, and is poorly commented. Bad code! Bad!

Another data resource

2011-04-18T12:03:00.000-07:00

I'm on a real data kick this week. Here's another one that should be useful for people training language classifiers.

lang_samples.tar.gz (1.5G gzipped)

This archive contains language samples from the 2008 static wikipedia dumps available at http://static.wikipedia.org/downloads/2008-06/. I downloaded all 261 archives, and extracted samples of text from each.

What does it take to be a data scientist?

2011-04-16T07:27:00.000-07:00

Conway on what it takes to be a data scientist (@ ZIA, ht: Mike B).

The full article is here. It's short and sweet, and offers a nice counterpoint to some of the claims made by people with a more computer-science-centric view of the world. Turns out that modeling assumptions (i.e. math and statistics) and theory (i.e. substantive expertise) matter. You ignore them at your own risk.

PS: The title makes it sound like this is about U.S. intelligence, but almost all the points in the article apply to business and academia as well.

More text archives

2011-04-15T12:18:00.000-07:00

Three more data sets (108MB, gzipped) for training classifiers. All these files are text-only, crawled in the week of 12/17/2010. It just took me a while to get around to releasing them.

political (115M):
Front pages from ~2,500 political sites. These sites were rated extremely likely to be political (p>.99) in an early version of my census of the political web. I revisited the same sites several months later and downloaded the new front pages to create this dataset. They should be appropriate for case-control training of a political classifier.

random (471M):
Front pages from ~8,000 "random" sites. These are a pseudo-representative sample of the web. I ran several web spiders in parallel, recording all outbound links from visited sites. I deliberately undersampled common namespaces, in order to avoid getting trapped in densely-linked social networking sites. The 8,000 sites are a random sample from a list of ~2 million sites generated in a crawl of this type.

porn (43M):
Front pages from ~1,200 pornography sites. I never thought I'd be curating pornography as part of my dissertation, but so much of the web is porn that I've had to build special classifiers to screen it out before sending to my undergrad research team for coding. These 1,200 sites were all linked from http://research.vision-options.com/research.php.

It's data dump week!

2011-04-14T12:07:00.000-07:00

In case this wasn't clear already, it's data dump week. I'm gearing up for another crawl of the political web, and posting lots of bits of code and data along the way. If nothing else, I'll be able to find them here in the future. If these (or similar) resources are useful to you, please let me know.

Cheers!

Data release: 75K political web sites

2011-04-14T08:28:00.000-07:00

Data release:
pol_site_sample.tar.gz (1.2GB, gzipped)

This archive contains results from a front-page-only crawl of ~75,000 political web sites on 4/9/2011.

These sites span a large portion of the political web, but they are not a representative sample. Each was classified as very likely to be political (p > 0.9) in a crawl performed in August 2010. In other words, these are pages from sites that featured political content 8 months ago. Presumably, most -- but certainly not all -- of the sites still feature political content today.

I'm going to use this to train content classifiers. Might be useful to others as well.

Links on crowdsourcing

2011-04-13T07:42:00.000-07:00

Point: Why I hate mturk (research)
Counterpoint: Managing crowdsourced human computation

Lightweight pdf renderers

2011-04-12T08:53:00.000-07:00

I'm finishing up dissertation data collection in the next ~6 weeks, which means I'm going to be spending a lot less time writing code, and a lot more time analyzing data and writing papers. So R and laTex are going to be my new best friends.

Taking a good look at my workflow around these packages, I realized that viewing pdfs was really slowing me down. Every time I generate a graph or paper, I have to open up the pdf version and see what it looks like. Adobe's very bulky software takes several seconds to load -- very frustrating when you're playing with margins or table formating and want to iterate quickly.

So I went out looking for a lightweight pdf viewer. Here's what I found:

http://www.downloadmunkey.net/2008/04/random-monday-foxit-reader-vs-pdf-xchange-viewer-vs-sumatra/
http://portableapps.com/node/17260
http://www.techsupportalert.com/best-free-non-adobe-pdf-reader.htm

Any other advice?

Based on those reviews, I'm going to give PDF-XChange a shot. I'll let you know how it goes.

Redesigned my layout

2011-04-11T07:40:00.000-07:00

I've been meaning to do this for a while. tpmotd's nudge finally got me to make the neccesary 15 minutes.

Now all my pictures should actually fit!

Prezi: AI and games

2011-04-09T10:48:00.000-07:00

A few months ago I first tried out prezi. Since then, I've seen it trickling into presentations here and there. The novelty is great, and the usability is a little better than last time I played with it. So I went ahead and

These are slides for my (mock) TED talk from the Hill Street TED activity yesterday.

Crowdsourcing and buzzword lumping

2011-04-07T07:14:00.000-07:00

In general, I'm a big supporter of crowdsourcing, but I worry about lumping together too many things under one popular buzzword. A few NYTimes articles have spoken to this issue recently (ht Gloria). This one is pretty starry-eyed. This one unpacks things (a little) more.

Let me push on this idea of lumping. Wikipedia defines crowdsourcing as

the act of outsourcing tasks, traditionally performed by an employee or contractor, to an undefined, large group of people or community (a "crowd"), through an open call.

As I read that definition, all of these are crowdsourcing:

Wikipedia
Ideo's open design lab
Innocentive's innovation contests
99design's graphic design sweatshop
Elections
Spam farms
Penny stock pump-and-dump marketing
Bounty hunters and privateers

Trick question: so is crowdsourcing a good thing or a bad thing?

My position: We're at a place where technology is enabling new institutions. It would be backward to ignore that potential. But there are all kinds of issues with corruption, lack of expertise, bias in who participates, etc. that "crowdsourcing" doesn't solve automatically. Just like other institutions, crowdsourced institutions have to be designed carefully to head off those problems. I don't think it's a magic bullet, but I do think it can help.

Announcing SnowCrawl!

2011-04-05T09:21:00.000-07:00

Announcing the beta release of SnowCrawl, a python library for directed webcrawls. Nice features include: saved state for backup, support for threading and client-server architecture, lots of flexibility.

The project is open source, hosted at google code. More details to follow.

Wanted: A mystery science theater for stats and programming books.

2011-03-14T09:08:00.000-07:00

Wanted: A mystery science theater for stats and programming books.

Looking for good introductory R books for non-programmers, (Know any? Books, I mean, not non-programmers.) I ran across this wonderful review of an apparently ludicrous book.

Statistical Analysis with R: Beginner’s Guide by John M. Quick

Summary: If you can get past the strange underlying story, then this gives a good introduction to R to someone with no programming experience.

[...]

[B]efore describing the data analysis section of the book, I should explain the underlying story used throughout the book. The introductory chapter gives a bit of ancient Chinese history, and states that you, the reader, have been chosen to succeed the famous military leader Zhuge Liang and need to learn how to use R to analyse his data and plan the future of the military campaign.

The rest of the book takes on this theme, both in the data analysis (comparing the Shu and Wei armies, and predicting battle outcomes using regression) and the general phrasing (headings like “Have a go hero!” and emphasis that if you fail the Chinese kingdom will collapse).

Wow.

Scalable models for education

2011-03-11T09:17:00.000-08:00

via my good friend Steve Stay:

The White House has a new initiative out: Winning the Education Future - The Role of ARPA-ED (16 pages)

To address the under-investment in learning technology R and D, the President’s FY2012 budget proposes to invest $90 million to create an Advanced Research Projects Agency for Education (ARPA-ED). ARPA-ED will fund projects performed by industry, universities, or other innovative organizations, selected based on their potential to create a dramatic breakthrough in learning and teaching.

ARPA-ED is trying to catalyze development of:
* Digital tutors as effective as personal tutors.
* Courses that improve the more students use them.
* Educational software as compelling as the best videogame.

The intuition here is really interesting -- building technologies for education that scale. Our ed system is based on a 150-year-old chalk-and-talk delivery system. One teacher, ~20 students, one lecture at a time, interspersed with homework, projects, and some small group discussion. I'm stylizing here, but you know I'm not too far off.

The major bottleneck in this system is effective teachers. Roughly speaking, the number of students getting a good classroom education is equal to the number of good teachers times twenty. Given the tremendous difficulty of recruiting, training, and retaining teachers, this is a serious constraint.

Therefore, I am a huge fan of the idea of deploying better education via scalable technology. But I don't think we've seen this done right yet. Some also-rans:

i-tunes U and similar platforms strike me as a partial first step -- maybe a good replacement for lectures. Suppose I'm a middling public speaker with reasonably good subject knowledge in an area. Why should I lecture when my students can hear from a top-of-the field virtuoso? But education is a lot more than content delivery. itunes can't give you feedback, answer questions, or hold office hours.

In the long run, private online universities may be able step up to provide scalable education. But so far I've been pretty unimpressed. For instance, University of Phoenix seems to be running a pump-and-dump model based on government subsidies through student loans. (Maybe I'm out of line on this claim, but that's my impression.)

I think we're still waiting for the real thing in scalable education. What possibilities am I missing?

Three cheers for crowd source/ open source

2011-03-09T16:12:00.000-08:00

Google is offering $10k in prizes for visualizations of the U.S. Federal budget.

R has overtaken Matlab and SAS in popularity

HP is now including a Linux-based OS with every PC it ships.

Information visualization and the Battle of the Atlantic

2011-03-09T09:24:00.000-08:00

I've been reading Churchill's 6-volume history of WWII. Fascinating reading if you're into this kind of stuff. Some scattered thoughts on history, war, and information:

WWII was arguably the first war fought through information as much as weaponry. One of Neal Stephenson's characters in Cryptonomicon has a great monologue on this point. He claims that Nazi Germany typifies the values of Ares (you know, the Greek god of war), and the U.S./U.K. typify the values of Athena. In this telling, WWII Germany had an advantage in guns and regimentation, but the proto-hacker cryptographers of Bletchley Park, etc. ran rings around them with information. I recommend the monologue, but not the whole book.

This comes out in Churchill's narrative. Exhibit A is a set of FlowingData-esque maps of merchant ships sunk by U-boats throughout the war.

A little background: in the middle part of the war (once France had been defeated, but before Russia and the U.S. had entered) the "Battle of the Atlantic" was probably the single most important "front" in the war. As long as England was connected to her colonies by convoys of merchant ships, she could continue to fight. If bombing and U-boat action could constrict this flow of trade sufficiently, the little island would have no chance.

Exhibit A: (scanned on the cheap with my pocket digital camera)

Charts like these make it clear that Churchill was interacting on wartime data on a day-to-day basis, and that that flow of information was crucial to war effort. Churchill likes to attribute success to the bulldog-like grit and willpower of the British people, but it's clear from his narrative that the flow of information was at least as important. In war, grit doesn't matter much without gunpowder.

In addition to maps, Churchill gives statistics and monthly trends for various gains and losses in shipping. They remind me of post-game trend plots in Starcraft II. The general tension between military and economy is the same. They also remind me of the dashboards that are all the rage in business process management these days. 50 years ago, you had to be a superpower at war to devote these kinds of resources to information gathering. Now, any reasonable-sized business has them. Heck, even this blog is hooked up to sitemeter. Map of the world, populated with little dots? Check.

AI for rock, paper, scissors

2011-03-08T17:10:00.001-08:00

A really simple and clever demonstration of another place AI is surprisingly good at beating people: a rock-paper-scissors game on NYTimes. Don't laugh. The computer analyzes your past play and looks for weaknesses to exploit. I played it 100 times and lost 23 to 35, with 42 ties. (I said don't laugh!)

Nifty application. Nice design. The "see what the computer is thinking" is a great way of explaining how the AI works. Much better than saying it's a 4th-order Markov process with backing off.

The small print on the side of the screen is revealing (emphasis mine):

Note: A truly random game of rock-paper-scissors would result in a statistical tie with each player winning, tying and losing one-third of the time. However, people are not truly random and thus can be studied and analyzed. While this computer won't win all rounds, over time it can exploit a person's tendencies and patterns to gain an advantage over its opponent.

This is nice evidence that people are surprisingly bad at being unpredictable.

PS. I ran the numbers. The odds of me doing so poorly against the computer in an even game (one third wins, one third losses, one third ties) are about one in 60, a p value of 0.015. Not impossible, but extremely unlikely. So the smart money is on the computer.

What examples for successful prediction do we have in social science?

2011-02-28T09:11:00.000-08:00

A question that's been bugging me: as social scientists, what can we predict reasonably well?

Following Kuhn's definition for a scientific paradigm, I'm focusing on:
1) social phenomena
2) that we can predict with enough accuracy to be useful
3) using technical skills that require special training.

I've found surprisingly few examples that satisfy all of these criteria. Only three, in fact.

Aptitude testing
Prediction: How well will a person perform in school, a given job, the Army, etc.?
Technical skill: Psychometrics

Microeconomics
Prediction: What effect will various market interventions have on the price, quantity supplied, etc. for a specific good?
Technical skill: Producer, consumer, and game theory

Election polling
Prediction: Who will win in a given election?
Technical skills: Survey design, sampling theory

Can you think of any others? I've got to be missing some.

What areas *should* we be able to predict? We have all kinds of new tools as social scientists. It seems like we should be ready to tackle some new challenges.

How to do research that has an impact, and still survive

2011-02-26T07:46:00.000-08:00

Fascinating talk by Richard Hamming, of hamming code fame: You and Your Research. It's a very long and detailed discussion of how to build a scientific career in a way that will maximize the long-term impact of your work.

And here's an interesting counterpoint from the other end of a scientific career: how to survive grad school. Some Modest Advice for Graduate Students.

Glenn Beck conspiracy generator

2011-02-21T09:10:00.000-08:00

A Glenn Beck conspiracy generator.

How does this thing work? I'm guessing mturk or some mailing list. The phrases don't seem quite formulaic enough for Markov generation or automated madlibs.

Best. Weather. Forecast. Ever.

2011-02-19T20:51:00.000-08:00

Weatherspark uses the same data as everyone else, but they make it so much more usable. Gorgeous interactive maps, trends, and predictions.

Link mishmash

2011-02-15T14:39:00.000-08:00

Here's some of what I've been reading/watching online lately.

1. From TED, on babies and how they "take statistics" to learn language. Super cute.

Comprehension question: at what age did you stop being able to understand Mandarin?

Presentation question: why does using FMRI/MEG/etc. increase one's credibility as a scientist? Several minutes of this 10-minute clip talk about brain activity and MEG output. But none of the important findings here are based on that data -- only speculation.

2. Via marginal revolution, a fair rent calculator.

This is for roommates deciding how much the big room with the extra window is worth. Based on this analysis, which uses surveys and hedonic regression to put numbers on preferences. (PS - "Hedonic Regression" would be a fantastic name for a nerd band.)

3. Two articles on credentialism in academia: from businessinsider, via Mike Bommarito; and from Steven Hsu, at Andrew Gelman's blog.

Interesting points made here. Has the value of formal, certified education increased or decreased? What about the quality of education? I've grown increasingly pessimistic about this topic of late.

4. And last, but not least a caption contest at wondermark (I love this web comic). Here's the pic. Submissions wanted.

How to win the final wager in Jeopardy

2011-02-15T06:29:00.002-08:00

Here's a fun sideline to the human-vs-AI Jeopardy showdown: optimal wagering strategies for final Jeopardy. The challenge: given what you know about yourself, your opponents, their bankrolls, and the question, choose the betting strategy most likely to secure a win or at least a tie. What do you do?

As it happens, some smart game theorists have already worked on this problem. Work in the 1990's shows that against your average human, it's not too complicated.

If you're winning, assume your opponents will bet everything, and react accordingly by betting enough to double the number two player's score, plus one dollar.
If you're in the middle it can be tricky, but a good bet is to triple your score and subtract double the first player's score.
If you're losing by a lot, bet everything.

Following these rules should increase your chances of winning, regardless of your starting position. (See pg 271 in the paper.)

Against a human (or computer) who knows some game theory, the ideal strategy is complicated, because your strategizing will prompt strategizing by your opponents, and vice versa, ad infinitum. The Gilbert and Hatcher paper gives a solution for the two-player scenario. This situation happens when one contestant has no money to wager in the final round -- it's actually not all that unusual.

The three-player scenario is unsolved, but it's almost certainly a mixed strategy. Sounds like a good problem for a different kind of AI: computational game theory. Given that simple versions of poker have been completely solved, a mixed strategy over wagers should be child's play.

With all that as background, here's IBM's page on how Watson wagers. They cite the ridiculously detailed Jeopardy folklore of betting strategies --- the entries for "two-thirds" and "Shore's conjecture" are particularly on point --- but evidently, they haven't worked through the game theory. Knowing this, Ken and Brad should be able to outflank Watson in the final round, giving them slightly better odds at cinching a win.

Punchline: in the world of AI, Watson is a verbal genius in need of some remedial math.

Realtime Congress API

2011-02-09T08:36:00.000-08:00

Just ran across this new API, for Congress, in real time. Put out by the Sunlight Foundation, an organization using "cutting-edge technology and ideas to make government transparent and accountable," the API allows you to track, floor actions, video, committee hearings, bills, amendments, votes, and other documents in real time.

Really looking forward to playing with this.

Hard problems in social science

2011-02-08T16:54:00.000-08:00

A Harvard conference on "Hard problems in social science" just wrapped up, with voting on the most important and most difficult problems. The ran the poll on facebook with 10,000 people weighing in.

Frontrunners include:

World peace
Sustainable population growth
Collective decision making
Stable institutions

The pdf with survey results is here. Videos of speakers are here. Very interesting stuff

Cracking the Ontario lottery - via Wired

2011-02-05T08:19:00.001-08:00

A great story on the statistician who cracked the Ontario lottery -- how he cracked it, why the lottery didn't believe him, how other lotteries are probably breakable as well, and why the mob likes it that way. A great read, all the way through.

Model Makers' Hippocratic Oath

2011-02-05T07:48:00.000-08:00

Model Makers' Hippocratic Oath:

I will remember that I didn't make the world and that it doesn't satisfy my equations.

Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.

I will never sacrifice reality for elegance without explaining why I have done so. Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.

I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.

Via Statistical Modeling, Causal Inference, and Social Science

QMD - A framework for prioritizing research ideas

2011-01-31T14:39:00.000-08:00

This from a recent exchange with a research assistant. He's trying to decide what to do for his undergraduate honors thesis, and I was trying to give him some sage advice. Here's the practical framework I use when deciding what research ideas to pursue:

Every research paper is a combination of three main parts: 1) a question, 2) a methodology, and 3) data. For instance, the Druckman paper [from this week's reading] asks 1) when elite framing can influence opinion using 2) an experiment where 3) college students read and discuss news articles. ...

(By the way, "data" includes any kinds of facts you'd use to build your argument, not just numbers. Surveys, content analysis, government statistics, experimental results, interviews, historical case studies, etc. are all data -- they're just analyzed differently.)

The combination of QMD that you choose largely determines how hard it will be to carry out the project, the kinds of conclusions you will be able to draw, and the threats to validity that critics can use against you. Choosing the QMD for a research project is like choosing a major in college -- in a lot of ways, all the later decisions are just details.

When you choose which Q, M, and D to use, you can either come up with something completely new, or borrow ideas from past studies. It is much, much harder to come up with new questions, methods and data than it is to borrow from past research. As a strong rule of thumb, you only want one of your three parts to be new. So if you're asking a new question, use familiar methods and easy-to-find data. (This isn't plagiarism because you're completely up front about the idea that you're building on previous work.)

Probably the easiest kind of study to pull off is a replication study, using new data with old methods and questions. For example, to deal with one of the threats to validity in Druckman et. al's study, you might do the same experiment with a representative sample of people, instead of just college students. Same question, same method, different data. This would still take work, but there aren't a lot of unknowns in the process, and anyone who believes Druckman and Nelson will probably believe your results as well. Replication studies are good for boosting ("Not only are Druckman and Nelson right about framing effects for college students -- they're right about all American adults as well!") or cutting back ("Druckman and Nelson's findings only apply to people under 30. Everyone else is much less susceptible to elite framing.") the scope of previous findings.

The next hardest is applying a new method to an old question. For example, you might ask when elite framing influences public opinion using a series of carefully timed surveys asking the same questions that Druckman et al used. Similar question, different method, similar data. This is a little harder, because you'd have to use different skills, build a new kind of argument, and deal with new threats to validity. On the plus side, different methods have different threats to validity, so if you use a new method, you can probably address some of the threats to validity that previous studies couldn't. For example, timed surveys could deal with the "recency effects" that Druckman can't do much about.

Introducing new questions is the hardest of all, for at least three reasons. First, you have to convince people that your question is worth asking. This is harder than it sounds. In my experience, people like to think they already understand the world pretty well, so they will resist the notion that you've found a blind spot in their worldview. Second, you have to be able to convince people that you're the first one to ask the question. Third, you have to build an entirely new line of evidence to defend your reasoning. With a one-semester deadline, I'd advise against trying to introduce a new question, just because it's so hard to do.

Anyway, that might be more than you want to know, but it's the way I decide what research projects to take on. QMD is my framework for figuring out what topics are in the overlap of "interesting" and "doable."

Thoughts?

Inspiring, hopeful, education, and salmon -- Not in that order

2011-01-27T17:32:00.001-08:00

This is everywhere already, so why not put it up here too? NPR asked people what they heard in the state of the union, then put the 4,000+ responses into a word cloud. Here it is.
Remind me to include a couple of jokes next time I give a talk.

Pennsylvania Policy Database Project

2011-01-27T07:39:00.000-08:00

I just ran across this project, with some interesting data sets on policy making in Pennsylvania. I'll come back later to investigate in more detail.

bubbl.us: A great little tool for flowcharts

2011-01-26T19:00:00.000-08:00

As I've gotten serious about this whole dissertation thing, I've realized that the process can get pretty complicated. "Before I can launch the survey, I need to draw the sample and write the questionnaire. But before I can write the questionnaire, I have to read the three papers that talk about measuring ideology, and ..." To cope, I've found myself drawing a lot of flowcharts to map dependencies.

Two days ago, I found this great little tool for flowcharts: bubbl.us. It's free, unless you want the premium license, and it has a really nice interface. Once you learn how to tab and shift-click and everything, you can sketch out complicated charts very quickly. If you allow it some room on your hard drive, you can even save maps between sessions. As I said, I'm using it primarily for mapping workflow, but I could see it being useful for brainstorming, simple org charts, or flow diagrams too.

And with that said, here's the map through the dissertation quagmire to the end of the semester.

An Interview with Ronald Coase

2011-01-13T09:45:00.000-08:00

A fascinating snippet of an interview with the famous Ronald Coase:

RC: Nothing guarantees success. Given human fallibility, we are bound to make mistakes all the time.

WN: So the question is how we can learn from experiments at minimal cost. Or, how could we structure our economy and society in such a way that collective learning can be facilitated at a bearable price?

RC: That’s right. Hayek made a good point that knowledge was diffused in society and that made central planning impossible.

WN: The diffusion of knowledge creates another social problem: conflict between competing ideas. To my knowledge, only people fight for ideas (religious or ideological), only people are willing to die for their ideas. The animal world might be bloody and uncivilized. But animals, as far as we know, do not fight over ideas.

RC: That’s probably right. That’s why we need a market for ideas. Ideas can compete; people with different ideas do not need to slaughter each other.

Really interesting tension. I'm still working through what to make of it. Collectively, we're smarter because we disagree? And vice versa?

Snipped here (Knowledge Problem just made my blogroll), with a hat tip to marginal revolution

Optimal Poker AI -- I told you so!

2011-01-01T09:31:00.000-08:00

A couple years ago, I had a long debate about Texas Hold'em with my brother, a long-time aficionado of the game. He's quite good and he's roomed with friends who actually payed the bills playing online poker. So you can see why he'd get defensive when I claimed that there is an optimal strategy for heads-up Texas Hold'em.

I've thought back to that conversation several times, mainly because it was really frustrating for both of us. I had a proof backing me up -- Von Neuman's 80-year-old minimax theorem* -- an absolutely airtight mathematical proof, but I still couldn't convince him. The problem is that it's an existence proof, so it demonstrates that the strategy must exist, but doesn't tell you what the strategy is. And of course, arguing that "my math says there's an optimal strategy" rings pretty hollow when you can't answer the question, "so what's the strategy?"

So I was very happy to run across this paper and demo presenting the optimal strategy for Rhode Island Hold'em. Rhode Island poker is a stripped down version of Texas Hold'em (get it?). You play with a smaller deck, only one hole card, a one-card flop, and a one-card turn, but no river. There are a few limits on betting, but the game is essentially the same. According the abstract, "Some features of the [optimal strategy] include poker techniques such as bluffing, slow-playing, check-raising, and semi-bluffing."

Very nifty proof of concept.

* A little bit more on the math. The minmax theorem states that every one-on-one winner-take-all game has an optimal strategy profile. A quick FAQ on what it means to be an optimal strategy.

So what does it mean that the strategy is optimal?
The technical definition is that it's an equilibrium strategy. If both players are playing the optimal strategy, neither has an incentive to deviate and play anything else. As one of my profs would say, it's prediction proof. Even if we both know each others' strategies, neither of us has an incentive to deviate.

So this strategy always wins?
No. There is some luck in the game. But if you played it repeatedly, there's no way you would win more than half the time. Almost any non-optimal strategy you'd play, the computer would beat you more than half the time.

Wouldn't an "optimal strategy" be really predictable?
No. In poker, it's often useful to bluff and keep your opponent guessing, so the optimal strategy is partly random. That means that when facing a given situation (e.g. After the flop, I have X cards, I did Y pre-flop, and my opponent did Z) the computer will flip a weighted coin to make its decision (e.g. 3/4 of the time I fold, 1/8 of the time I raise, and 1/8 of the time I check.) So the strategy is perfectly well-defined, but still unpredictable at times.

Does it know how to play short-stack and long-stack poker?
I don't think so. As far as I can tell, Rhode Island poker is played one hand at a time, with limits on betting. So the optimal strategy doesn't need to know anything about stack length, escalating blinds, going all in, etc.

If you introduced those elements, you'd be playing a different game. According to the min-max theorem, there would still be an optimal strategy, but it would probably be at least a little different.

A graphic design challenge...

2010-12-20T05:54:00.000-08:00

Next semester, I'm going to be conducting a large-scale survey of bloggers for my dissertation. (I've had a pilot wave in the field for the last several weeks.) For credibility's sake, I need a public face for the project -- a web site people can visit to reassure themselves that my survey request isn't some kind of phishing scam.

I'm branding the project as the Online Political Speech Project, with the goal of "understanding the voices of the Internet." I've worked out some rough ideas for visual motifs, but I'm no graphic designer. If you are, or know someone who is, I'd love to be in touch. I'm hoping this can be a fun project for some talented graphic designer...

Project specs: I need to design and lay out this web page by the second week in January. I'm imagining about five pages (splash page, FAQ, contact, blog, data repository). Scope of work would be limited to designing templates -- I can plug in content later.

This should be reasonably quick work, with a lot of creative control for the designer. I can offer a little cash, but probably a lot less than going professional rates. (That's because, as a grad student, I get paid a lot less than going professional rates, too.) Maybe a good portfolio piece for a student web designer...? Or else a fun project for someone who wants to play with some of the styles and themes that have emerged lately on the web...?

Please email me (agong at umich dot edu) if you're interested.

Google's n-gram viewer - pros and cons

2010-12-17T06:42:00.000-08:00

Google just released an n-gram viewing tool for tracking trends in half a trillion words across 400 years of text. Check it out. It's super fast and interesting, kind of like Google trends on timescale steroids. Even better, they released the data.

Assorted thoughts poached from conversations with friends:

Someone should tell Fox about "Christmas"

I predict that in the year 165166 AD, fully one hundred percent of the words written in English will be the word "dance." (This will make 500-word APSA applications much more straightforward.)

My take
(In 18 words: For scientists, a small, representative sample usually beats a large, non-random sample. Google has given us the latter.)

The n-gram viewer will be a lot of fun for journalists, who often want a quick graph to illustrate a trend. It's a gold mine for linguists, who want to understand how syntax and semantics have changed over time. For both of these fields, lots of data equals better.

As a bonus for me, maybe this tool will popularize the term n-gram, so I won't have to keep explaining it in conference talks.

I'm not sure about the impact the n-gram viewer will have on the areas where the researchers seem to want to apply it most: culture, sociology, and social science in general. The reason is that in those fields we tend to care a lot about who is speaking, not just how much speech we have on record. This is why the field of sampling theory has been a consistent cornerstone in social science for almost a century. From the press releases, we can see Google's claim that this is 5.2% of all books ever written. But we don't know if it's a random sample.

And in any case, books have never been written by a random sample of people. I suspect that the n-gram viewer will have very little to offer to researchers hoping to study the culture and sociology of, say, blacks in the Jim Crow south, or the early phases of Lenin's revolution. By and large, we already know what cultural elites were saying in those periods, and the rest weren't writing books.

This means that there are two filters in place for any social scientist who wants to pull cultural trends out of these data. First, there's Google's method for sampling books, for which I haven't seen any documentation. (No offense, but this is pretty typical of computer scientists, who think in terms of scale more than sampling). Second, there's the authorship filter: you have to keep in mind that any trends are derived from written language, produced by whatever subset of the population was literate in a given period

Example
As a political scientist, I'm interested in conflict. If you go to the site and punch in "war," from 1500 to 2000, you get a graph showing quite a few interesting trends. Here are some stories that I could naively tell from this data. I suspect that many are false.

In general, humanity has been far more interested in war since 1750.
That said, interest in war has generally declined since then.
Interest in wars spikes during wars.
Proportionally, wars since 1750 affect people far less than they did previously -- the use of the term "war" jumped up to 5 times the period baseline during the English civil war, but less than double the period baseline during the World Wars.
World wars I and II were the historical high points for human interests in war, but only slightly higher than the sustained interest of the peak period in 1750.

Interesting conjectures, and I can spin a story for each of these. In several cases, I'm pretty sure we're seeing the effects of cultural or linguistic artifacts in the data. Potential confounders include: changes in literacy rates, introduction of public schooling, the American revolution, improvements in printing technology, linguistic drift, etc.

I'm cautiously pessimistic here. Given all the potential confounding variables, it's hard to see how we can sort out what's really going on in the line graphs. Maybe we can do it with fancy statistics. But we're not there yet.

I'll give the last word to BJP:

My instinct suggests that what is talking here is less the folds of culture and more the wrinkles of the record of the technology expressing that culture.

Link mish-mash

2010-12-16T05:42:00.001-08:00

All kinds of interesting stuff on the web today. Not that this wasn't also true yesterday, or the day before...

IBM says its Watson AI is ready to take on human Jeopardy champs for a $1 million prize. The showdown is scheduled for Feb 14. This is reminiscent of the Kasparov/Deep Blue showdown, except that Watson will be competing on human home turf: making sense of the linguistic ambiguity in the hints, phrasing, puns, etc. of Jeopardy prompts. (The AI has one advantage: I bet Watson will always remember to phrase its answer in the form of a question.)

A rehash of physicist Aaron Clauset's work on "the physics of terrorism." I'm not a big fan of his stuff, to be honest. My view: Clauset showed that the severity of terrorist attacks follow a powerlaw distribution, and has been wildly extrapolating from that single finding ever since.

A Jeremiad about the state of journalism, from Pulitzer prize winner David Cay Johnson. He talks trends (reporters know less and less about government; papers keep cutting content and raising prices), and hints at causes. The last paragraph is especially intriguing to me. Read it as a claim about how good reporting is supposed to uncover truth.

Predictions about 2011 from 1931. Eighty years ago, the NYTimes gathered a brain trust of experts in various fields and asked them what the world would look like today. Follow the link for predictions and some commentary. (Hat tip Marginal Revolution.)

This paper should depress my libertarian friends. Evidently, profit is evil, with an r-value of -.62.

The best data visualization projects of 2010, from FlowingData. Beautiful and informative.

HPN health prize: $3,000,000

2010-12-13T16:48:00.000-08:00

The Heritage Provider Network is offering a $3 million prize for predicting hospitalization -- which patients are likely to be hospitalized or rehospitalized?

In 2006 well over $30 billion was spent on unnecessary hospital admissions. How many of those hospital admissions could have been avoided if only we had real-time information as to which patients were at risk for future hospitalization?

Programming Language Popularity Contest

2010-12-11T12:36:00.000-08:00

I keep telling people that python is the language to learn. Here's proof. Half a million github and StackOverflow programmers can't be wrong, right?

(Ignore javascript. It's an in-browser language only. Fun if you want to animate a web page; useless if you want to write a file.)

via : Programming Language Popularity Contest

Journalism and activism: Stylized facts

2010-12-07T12:32:00.000-08:00

Trying this idea on for size. Reactions?

(Also, I'm attempting to post to facebook via Notes. Let's hope the feed actually works now...)

The United States of Autocomplete

2010-12-06T07:13:00.000-08:00

Hat tip FlowingData: The United States of Autocomplete

Very Small Array has some fun Google's autocomplete. Utah... Jazz. Kentucky... Fried Chicken. New York... Times.

[Very Small Array via @mericson]

Arsenic-based life and autocatalytic sets

2010-12-02T18:15:00.000-08:00

NASA has just announced the discovery of a new kind of extremophile bacteria. If I'm reading the (sketchy) early release info correctly, these critters use arsenic in their DNA, instead of phosphorus. This is a biggish deal because every other known life form uses phosphorus.

How big a deal? Setting aside shades of the Andromeda Strain*, people seem pretty underwhelmed. Partly it's a bait-and-switch involving the NASA brand: the microbe in question, Gammaproteobacteria GFAJ-1, isn't an alien visitor. It's a mutant strain of a run-of-the-mill terrestrial bug.

So the real question is: How surprised should we be that little GFAJ-1 managed to assimilate arsenic into its DNA? Or the converse: if it's so easy, should we be surprised that no other life form bothered to do the same thing?

Taking a conversational step sideways, this seems like a good moment to put in a plug for a really fascinating theory on the origin of life: autocatalytic sets. This theory--which I find persuasive--argues that life isn't rare or unexpected -- it's virtually inevitable. I can't find a good non-technical description of this underappreciated set of ideas online, so I thought I'd take a crack at it here . (My explanation is based largely on the opening chapters of Kauffman's At Home in the Universe).

The puzzle: all forms of life we know of are pretty complicated. There are a couple hundred cell differentiations in a human body; thousands of enzymes in a working cell; millions of base pairs in even the simplest DNA. (Prions and viruses, which are often simpler, don't count because they can't replicate without hijacking a cell's machinery.) With these kinds of numbers, the odds of getting the just right combination for life are astronomical. Try flipping 220 million coins until they line up with the base pairs in a single human chromosome. Practically speaking, it will never happen.

For most explanations for the origins of life, complexity is a stumbling block. We just got lucky, or God intervened, or maybe an asteroid seeded the planet to produce something capable of self-replication. There's a leap in the logic at the point where we try to explain the incredibly low probability of life emerging spontaneously from a lifeless chemical soup.

The autocatalytic theory turns this logic around: it argues that life exists because of its chemical complexity, not in spite of it.

The theory builds from three simple assumptions. First, some chemicals react; most don't, with roughly constant odds for any given pair of chemicals reacting. Second, chemical reactions produce new chemicals. Third, the number of possible chemicals is very large.

Thought experiment: suppose you find yourself in front of chemistry set containing a rack of n beakers, each filled with an unknown chemicals. Channeling your inner mad scientist, you begin mixing them at random. Suppose 1 in 100 pairs creates a reaction. How many total reactions should you be able to get out of your chemistry set?

If you have two chemicals, the odds of a reaction are just 1 in 100.

If you have three chemicals, you have three potential pairs, and the odds of a reaction are about 3 in 100.

If you have four chemicals, you actually have six potential pairs, so the odds of a reaction are a little better than 6 in 100.

At this point, exponentials start working rapidly in your favor. With five chemicals, you have 10 potential pairs, for a 9.5% chance of at least one reaction. Twelve chemicals gets you 66 pairs with 48.4% odds of at least one reaction. The deluxe 30-chemical starter kit has 435 potential pairs, with 98.7% odds of at least one reaction.

What does this prove? The number of likely reactions in a pool increases faster than the number of chemicals in the pool.

It keeps going, too. With 1 in 100 odds, you would expect to get about 4 reactions out of your 30-chemical kit. If each reaction creates a new chemical, you now have 34 chemicals in your pool, with correspondingly greater odds of additional reactions. Eventually, you pass a tipping point, and the expected number of compounds becomes infinite. If I've got my math right, it happens around 80 chemicals in this scenario, because the expected number of new reactions exceeds the number of reactions in the existing set. The more you mix, the more potential mixtures you get.

A quick pause: when we talk about chemicals, we're not talking about atoms in the periodic table of the elements. Except during fission and fusion, atoms themselves don't combine and react. Instead, we're talking about molecules.

In particular, organic molecules -- the ones that ended up supporting life -- fit this model very well. Enzymes, RNA, and DNA are all organic molecules. Believe it or not, a strand of DNA is one enormous molecule. Organic molecules are all built mainly from the same base atoms: carbon, hydrogen, oxygen, nitrogen, phosphorus -- and now arsenic(!) These atoms happen to be good at linking to form long chains. Because of the way these chains fold, they react with each other in often unpredictable ways. Most organic molecules don't react with each other, but quite a few do. And because the chains can get very long, the set of potential molecules made of CHONP atoms is essentially infinite.

Now getting back to the main story... We're halfway through. We've shown how simple rules for reactions can get you from a small-ish starting set to an infinite variety of chemicals to play with. It seems very reasonable to suppose that the primordial organic soup included enough organic reactants to pass the tipping point into infinite variety. But that just means a more flavorful soup. How do we get to life?

Setting aside the transcendental, life is defined by sustainable reproduction. A cell is a bag of chemicals and reactions that keeps working long enough to make at least one copy of itself. As part of the deal, the cell has to be in the right sort of environment, with whatever energy sources and nutrients are necessary.

Our cells achieve sustainability by using enzymes to catalyze other reactions. It turns out that the same logic that applies to pairs of reactions also applies to catalyzers: the probability of catalysts in a pool increases faster than the number of chemicals in the pool. Once you get enough chemicals, it's virtually certain that you'll have quite a few catalyzed reactions.

Here's the really nifty bit. As the number of catalyzed reactions in the set increases, eventually some of them will form an autocatalytic set -- a loop of reactions catalyzing each other. Reaction A creates the catalyst that enables reaction B, creating the catalyst for C, and so on back to catalyst Z that enables reaction A.

Based on the same logic we saw earlier, these loops always appear once the pool of chemicals gets large enough. They are typically long and complicated, cycling through a seemingly random group of chemicals among a much larger set of nutrients and byproducts. They tap nutrients and energy sources in the environment, increasing themselves the longer they run. In other words, autocatalytic sets look a whole lot like life as we know it.

I find this theory compelling. It takes the biggest objection to prevailing theories -- the inherent complexity of life -- and makes it the cornerstone of a different approach.

And as a bonus, it makes arsenic-based life forms seem very plausible. Given NASA's results, it seems reasonable to say that arsenic-based DNA is another unexplored evolutionary path for viable autocatalytic sets. Bill Bryson says it well in A Short History of Nearly Everything:

"It may be that the things that make [Earth] so splendid to us---well-proportioned Sun, doting Moon, sociable carbon, more molten magma than you can shake a stick at and all the rest---seem splendid simply because they are what we were born no count on. No one can altogether say. ... Other worlds may harbor beings thankful for their silvery lakes of mercury and drifting clouds of ammonia."

PS: Autocatalytic sets don't have much to do with the evolution/intelligent design debate. They propose a mechanism that could be responsible for jumpstarting evolution. So if you're comfortable with the idea that God would choose cosmological constants and direct evolutionary processes with some goal in mind, it probably won't bother you to add the idea that he would use chemical soups and catalysis networks along the way.

PPS: The main difference between an autocatalytic set and life as we know it is the absence of a cell wall. It's not hard to close the gap conceptually. Once a catalytic loop gets started, other loops usually form as well. At this point, competition and natural selection between autocatalytic cycles can kick in. If one autocatalytic loop happened to produce a hydrophobic byproduct (like a fat or lipid), it could easily act as a primitive cell wall. This kind of barrier would enable the autocatalytic loop to increase its concentration, and therefore its reaction rate. This kind of pseudocell would reproduce faster and very likely evolve into more sophisticated organisms.

*A ten word review of the Andromeda Strain: Typical Crichton--some interesting ideas; mildly annoying narration; mostly plotless.

** A wonderful book for putting some color on the messy process of scientific discovery.

Prezi - A quick review

2010-12-01T08:00:00.000-08:00

I spent a couple hours playing around with Prezi, an online presentation builder being promoted as an alternative to the slideshow format of powerpoint, keynote, etc. Instead of a series of slides, Prezi presentations consist of a series of views over one large image. The format parallels drawing on a whiteboard, instead of clicking through slides on a projector. A good concept, but the execution is a little clunky.

My review: Preparing good presentations is time consuming, for two reasons: 1) it takes some trial and error to figure out the best way to express an idea, verbally and visually, and 2) presentation software is clunky, requiring a lot of fiddling to get things right. In my experience, spending time on (1) is fun and creative; spending time on (2) is frustrating and stressful.

Because of the "whiteboard" metaphor and brand emphasis on good design, I was hoping Prezi would deliver a slick and streamlined user experience. Being free from interface hassles and able focus on creative expression would be wonderful. Alas, I quickly ran into many GUI annoyances.

The interface for importing images is very clunky. You have to download or save the image to your desktop, then upload. On the plus side, you can batch upload several images at a time.
The whole image is static, which means that you can't mark up images over the course of a presentation. To some extent this makes sense -- dynamic images would mess up the concept of arranging your display in space rather than time. However, it breaks the whiteboard metaphor. When I do whiteboard presentations, I often have an agenda that I revisit, adding checkmarks and lines to relevant content. I can't do that in Prezi.
Rudimentary tools for grouping object are not available. This one really gets me. You can accomplish the same thing (visually) by putting several objects together in an invisible frame. But every time you want to move the group, it takes several extra clicks to select everything and drag it around. Poor usability.
You can only use a handful of presentation styles. Your only alternative is to hire Prezi staff to build a custom style for $450.

Summary: I would really like a tool that lets me express myself clearly, fast. Prezi offers some advantages for clarity, but not really for speed. Overall, I'm mildly impressed, but not overwhelmed. For the moment, the main benefit of Prezi seems to be novelty.

Forums and platforms for crowdsourcing

2010-11-30T08:07:00.000-08:00

I've always thought of crowdsourcing as a narrow, niche thing. But then I ran across this excellent list of forums and platforms for crowdsourcing. Skimming down the list, I realized that crowdsourcing is actually starting to play an important role in quite a few industries.

It looks like we're just starting to climb the adoption curve here. Where should we expect crowdsourcing to stop? What sorts of problems will it solve (and not solve)? Who won't ever use it?

PS: In case you haven't run across the term before, here's wikipedia's definition of crowdsourcing:

Crowdsourcing is the act of outsourcing tasks, traditionally performed by an employee or contractor, to an undefined, large group of people or community (a crowd), through an open call.

How long are books in political science?

2010-11-24T06:53:00.000-08:00

As a polisci student who has passed his prelims, I have a very good idea how long books take to read. But I realized this week that I didn't have a good idea how long books take to write.

As I was setting up the template to write a book-format dissertation, I decided to give it a look. Here are some rough stats for five well-known polisci books*, pulled from my reading shelf more or less at random:

	Hindman	Zaller	Mutz	Lupia and McCubbins	Huber and Shipan
Lines/page	38	45	35	42	39
Words/line	13	13	11	12	11
Total pages	141	310	125	229	210
Total chapters	7	12	5	10	8
~Pages/chapter	20.1	25.8	25	22.9	26.2
~Total words	69,654	18,1350	48,125	11,5416	90,090

These estimates are probably biased upwards, because I based them on pages of full text without any tables, graphs, or chapter breaks. But still, it's a pretty good idea for the rough scale of the project.

To put it in perspective, the current working draft for my dissertation includes 10,653 words. If I'm shooting for 70,000 total, that means I'm 15% done already!

*These are the books I used for estimates:
Matthew Hindman, The Myth of Digital Democracy
John Zaller, The Nature and Origins of Mass Opinion
Diana Mutz, Hearing the Other Side
Arthur Lupia and Matthew McCubbins, The Democratic Dilemma
John Huber and Chuck Shipan, Deliberate Discretion

An update on voice recognition software

2010-11-23T07:15:00.000-08:00

Thanks to everyone who sent comments about voice recognition via Dragon, Microsoft, and Mac. I'm still playing with different options to see what will work.

In the meantime, I've discovered another option: Google Voice. It turns out there's an easy way to set up Google Voice so that you can dictate messages to yourself (info here). GV will then automatically transcribe them and send them to your email, or mobile device. Nifty!

Since it's free and didn't require any software installation or training, I decided to give this a shot first. The result: laughably bad transcription (see below), but a substantial boost in productivity. How's that? For me, drafting is the most time-consuming part of writing. Once I have some basic ideas on paper, I can edit and elaborate reasonably quickly. But it takes me a long time to get that first version out.

As a result, dictating an early version has been very helpful. It forces me to say something. Since I'll be editing soon, it doesn't matter if that something is bad (it is, usually). So dictation, even with GV's horribly inaccurate transcription has worked pretty well for me. Bottom line: I'm certainly going to be investing in a voice recognition package in the near future.

An example of GV transcription. Here's what I said:

This chapter describes methods and data sources for the book. My goal is to describe the logic behind the research design. The focus here is validity: what types of conclusions can we draw from these data? Technical details---of which there are many---are saved for appendices.

Here's what Google thought I said:

This chapter just tries messages and data sources for the book. My goal is to describe the logic behind the research design. The focus here is validity. What types of conclusions. Can we draw from the Yeah, technical details. I wish there are many her saved for the appendicitis.

In fairness, Google just got into this business in the last year or so. I'm sure their transcription will get better over time. But for the moment, "I wish there are many her saved for the appendicitis."

Anybody have any experience with dictation software?

2010-11-20T18:09:00.000-08:00

Faced with writing a 200-page thesis in the next year, I'm toying with the idea of getting myself a voice recognition software package. Dragon seems to be the industry leader. Here's a demo.

The software seems to work, is getting good reviews on Amazon, and is not terribly expensive. On the other hand, it seems like it might be hard to find a place to talk loudly to oneself and a computer for long periods of time. It also seems likely that training the software and editing its mistakes could be pretty time consuming. Also, Dragon seems built with MS Office in mind. Given that I lean open source (and don't even have Office on my laptop), would it work for me?

Bottom line: would this be more like the digital stylus I got a few months ago (and use all the time)? Or the extra external hard drives I got a year before that (and have only booted up to make sure they work)?

So I'm on the fence. I really wish Dragon had a trial version. Anybody have anything to add here?

PS: Yes, I do know that one episode of The Office. (Dwight: "Cancel card. Can-suhl car-Duh.")

PPS: How long do you think until a viable open-source option opens up? Probably at least a couple years, unless someone like Google decides to launch a free version. They seem to be making moves in that direct (see here for an easy application, and here for a broader one).

Training a text classifier

2010-11-19T10:55:00.001-08:00

I've been doing a lot of work with text classifiers lately. (See this post for an example.) It's an interesting process blending intuition, text, and math in some nifty ways.

Anyway, I was going to spell out how it all works in a blog post, but I got side tracked making an illustration, and now I'm out of time for blogging. But I like the illustration. More to come...

TED talk: Teaching kids real math with computers | Video on TED.com

2010-11-17T13:58:00.000-08:00

Conrad Wolfram: Teaching kids real math with computers | Video on TED.com

I like this.

Starting to get some dissertation results...

2010-11-04T18:59:00.000-07:00

Apologies for the long delay between posts. Stock excuse: "Dissertation... blah blah blah..."

Actually, I'm starting to get some nifty results from my dissertation. I've spent a long summer writing surveys and software, and in the next few weeks I hope to have something to show for it. Exhibit A: a word cloud for an automated classifier of political content.

Orange words are associated with political content, and blue words are disassociated. The size of a word denotes the strength of association -- essentially, the size of each word corresponds to the absolute value of the beta value of the word in a logistic regression with "political-ness" as the dependent variable. The layout of the words is done by computer algorithm to conserve space; it doesn't carry any important information.

I used wordle for the layout. The classifier runs regularized logistic regression using the scikits.learn package for python. The training data is from a team of undergraduate research assistants.

Daily Beast's Election Oracle

2010-10-08T15:59:00.000-07:00

Here's a nifty new application in sentiment analysis: the Daily Beast's "Election Oracle." It's based on an engine called WiseWindow, which apparently scrapes the Web in real time, searching for content relevant to all the midterm elections, then makes predictions. One interesting touch is that DailyBeast pipes in RealClearPolitics' latest poll numbers, inviting head-to-head comparisons.

If the screen-scraping works the way it's intended, you would expect WiseWindow's numbers to change a little faster than the polls, since polls are only conducted every so often, and usually include a rolling average.

I'm skeptical. Not because I don't believe that it can be done, but because I've read up on the top-of-the-line opinion mining algorithms, and they're still full of holes. Computer scientists are smart people, but they haven't absorbed the lessons that pollsters have spent almost 100 years learning.

My particular brand of procrastination

2010-10-06T17:16:00.001-07:00

On procrastination

2010-10-05T05:42:00.000-07:00

"Victor Hugo would write naked and tell his valet to hide his clothes so that he’d be unable to go outside when he was supposed to be writing."

If only I didn't have to do so much work from the CAP lab...

Here's a great, well-sourced article in the New Yorker on the psychology, philosophy, economics, and history of procrastination. Worth a read for all grad students.

A minute to learn, a lifetime to master

2010-09-16T17:05:00.000-07:00

My family was in town last weekend (new baby--very exciting) and my brother Sam and I got to talking about board games, and what makes a good game. One reasonable standard comes from Othello's tagline: "a minute to learn, a lifetime to master." For strategy games, at least, we like the idea of complex gameplay emerging from simple rules.

Following that idea, we pulled up a list of ~30 well-known games and rated them for "game complexity" (How complicated are the rules?) and "strategy complexity" (How complicated are strategies for winning?). We came up with our ratings separately, but they were strongly correlated (R-squared values of .68 and .73, respectively).

The strength of this correlation surprised me at first. In retrospect, board games have been such an important part of brotherly bonding that I shouldn't have been taken aback. Case in point: we had agreed to use 1 to 10 scales, but we both independently decided that War and Candyland should score zero for strategy complexity, because neither game ever presents players with a choice.

With data in hand, I averaged our scores and then plotted game complexity against strategy complexity. The two dimensions of complexity are correlated (R2 = .494). Note that both my favorite game (Go), and his favorite game (Texas holdem) are strong positive outliers.

Apologies for the ugly graph. If I had the time, I'd have plugged in logos for all the games. Still, the picture mostly comes through.

Link mishmash

2010-09-08T13:55:00.000-07:00

Interesting post by Dan Hopkins on the Monkey Cage: Spin and the economy -- Can newspapers convince people the economy is better or worse than it really is? No.

Also from the MC: predictions for the 2010 elections. We're short of complete consensus, but the Dems are likely to lose the House.

From FlowingData: a beautiful delayed-shutter picture of fireflies in a forest. See also the delayed-shutter picture of a Roomba.

From NYTimes: folk wisdom about study habits -- some of it masquerading as scientific -- is often dead wrong. This whole article is worth reading, but let me spoil a few of the punch lines. Designated study rooms -- bad. (What about designated study times?) Learning styles (e.g. "I'm a visual learner") -- very little supporting evidence. Testing -- often beneficial.

An xkcd wedding cake. I love this web comic.

Please link/subscribe to me!

2010-08-18T19:05:00.001-07:00

A cordial request: if you read or feed this blog, please link, subscribe, etc. in whatever format is right for you. I'd like to bump up my PageRank etc. a little. I promise I won't do this often.

Thank you! Here's a puppy:

We now return to our regularly scheduled social/computer-sciency blogging.

Probabilistic computing?

2010-08-17T11:26:00.000-07:00

Hat tip SMCISS.

For over 60 years, computers have been based on digital computing principles. Data is represented as bits (0s and 1s). Boolean logic gates perform operations on these bits. A processor steps through many of these operations serially in order to perform a function. However, today’s most interesting problems are not at all suited to this approach.
Here at Lyric Semiconductor, we are redesigning information processing circuits from the ground up to natively process probabilities: from the gate circuits to the processor architecture to the programming language. As a result, many applications that today require a thousand conventional processors will soon run in just one Lyric processor, providing 1,000x efficiencies in cost, power, and size.

Hat tip SMCISS.

Psychologists say that power corrupts, via WSJ

2010-08-15T08:05:00.000-07:00

Psychologists say that power corrupts, via WSJ. I know that this thesis is pretty widely accepted, but I want to snipe at it. I think we may be mistaking correlation for causation. The general arc of my argument is that thuggishness may (sometimes) lead to power, instead of the other way around.

First, all the cited psych studies showing "nice guys finish first" are from areas with dense social networks (dorms, sororities, chimp clans) and repeated person-to-person interactions. Not surprisingly, likability translates to popularity here. But I don't see any particular reason why that conclusion should translate to accumulation of power through job interviews or elections. I'm going out on a limb here, but perhaps the qualities that make frat boys popular are not the same qualities that put CEOs in power.

Second, at risk of alienating a lot of my friends in political psychology, I'm not a big fan of gimmicky priming studies. For instance: tell me about a time when you felt powerful, then write the letter E on your head. You wrote it backwards? See -- power corrupts.

Err... run that by me again? At what point does this experiment measure "power," and "corruption," and where does it demonstrate a casual relationship between the two? Sure, the researchers who did this study can do a song and dance about it, but at the end of the day, there are a lot of leaps in the logic. The study is gimmicky. Interesting maybe, but nowhere close to conclusive.

I completely accept the notion that our minds move between different mental states and that primed manipulations can nudge us from one state to another. But I'm not persuaded that we've mapped the set of states and priming cues in enough detail to draw general conclusions -- particularly when treatments and responses are measured in ways that never occur in normal human experience.

Third, the idea that transparency might fix things is nice, but lacking evidence. This is a place where careful study of incentives and strategy could add a lot of leverage. Go game theory!

Parting shot: I'm not anti-niceness or pro-corruption, but I think we often demonize authority and authority figures. There are an awful lot of social problems that best resolved through the benevolent use of authority, and I know plenty of people who live up to that charge. To the extent that power has a corrupting tendency, we should be looking for, promoting, and celebrating those who are proof against it.

Wacom tablet: post-purchase rationalization

2010-08-14T14:05:00.000-07:00

So I bought a Wacom Bamboo tablet last week. It's a nice digital pen tablet, with a drawing area about the size of a postcard -- a good size for sketching & gesture-based interfaces. It plugs into a USB port and (after some fiddling with the drivers) works very nicely with Windows, Firefox, Inkscape, etc.

That said, I'm not really sure why I bought the thing. I do a lot of writing and programming, which leads to a lot of typing, but not much clicking and dragging. Sure, the touchpad on my notebook is small, but it's not really $60 small. I blame postpartum sleep deprivation. It might also have something to do with watching this TED talk.

Anyway, in an effort to assuage my post-purchase cognitive dissonance, I've been telling myself that if I can improve the speed and accuracy of my clicks by just a fraction, this tablet thing will easily pay itself off in productivity in the long run. Right?

To bring some proof to that claim, I dug up this flash-based target practice game. Little targets fly around the screen and you try to click on them: score points for every target you hit; bonus points for consecutive bulls-eyes; miss too many times and the game is over. Great sound effects. This is high brainpower stuff.

I played three rounds in time trial mode, using the touchpad. Mean score 438. Then I played thrice with the tablet. Mean score 685!

To make sure this wasn't just an improvement in my reflexes and strategy (shoot the spring targets at the apex; don't waste a shot on a yo-yo target that might be yanked), I employed an interrupted time series design and played six more rounds. Mean score with the touchpad: 407. Mean score with the tablet: 977!

With that kind of performance improvement, the tablet was clearly worth it. Minority report, here I come.

TED talk on origami

2010-08-12T11:10:00.000-07:00

James Lang, physicist and origami master, speaks at TED. He talks about a fascinating intersection of art and math: origami.

This is a great, visual talk that links ideas you would never expect to come together: water bombs, ibex, and spiders turn into satellites, heart valves, and airbags, using the math of trees and circle packing. A really great example of the unexpected practical value of mathematics.

Hat tip to my brother Matthew for putting me onto Lang's work.

P != NP? The biggest mathematical problem in computer science may be solved

2010-08-09T05:02:00.000-07:00

Banks, businesses, and credit card users can rest easy: public key encryption is still secure. Mathematicians breathe a sigh of relief: computers aren't going to replace you by churning out brute force proofs. Vinay Deolalikar, a researcher at HP Labs, is claiming that he has verified the most important conjecture in mathematical computer science: P != NP.

Some links, and then onto an explanation of why this is all a big deal.

Greg Baker broke the story.
The slashdot squib.
Here's a very mathy site with some good context and historical background.
Here's a wiki section on consequences of a proof one way or the other. I don't like it much, so I'm going to try to do better here.

What's the difference between P and NP?
Suppose you have a friend -- a real whiz at math and logic. "I can answer any question you ask*," she promises, "it just might take some time." Before you ask her to solve your sudoku, route the world's airline traffic, or crack your bank's electronic encryption key, it's worth figuring out "Just how long is it going to take?"

For many kinds of problems, the answer depends on how many moving parts there are. To cite a famous example, suppose I'm a traveling salesman, using planes, trains, and automobiles to get between cities. I have n cities to visit. What is the fastest route that lets me visit every city?

Intuitively, the more cities there are, the harder this problem is going to be. If n=1, the answer is trivial. I start in the single city and just stay there -- no travel time. If n=2, the answer is easy. It's equal to the amount of time between the two cities (or double that, if I'm required to make a round trip.) If n=3, it starts to get complicated because I have several paths to check. If n=100, then solving the problem could take a long time. Even your math whiz friend might have to chew on it for a while.

In the world of computer science, P-type problems are the fast, easy, tractable ones. They can be solved in "polynomial time": if the problem has more moving parts, it will take longer to solve, but not a whole lot longer. Mathematically speaking, a problem belongs to complexity class P if you can write a function f(n) = n^k and prove that a problem with n elements can be solved in f(n) operations or less. Constructing proofs of this kind is a big part of what mathematical computer scientists do.

If we could prove that k=3 for the traveling salesman problem -- we can't, by the way -- then we could guarantee that a computer could solve the 100-city problem in 1 million time steps or less. If your math whiz friend uses an Intel chip running millions of operations per second, that doesn't look so bad. For P problems, we can put a bound on running time, and know that as n increases, the performance bound increases at worst by the exponent k.

NP-type problems can get harder, because the definition is looser. Instead of requiring your friend to come up with the answer in polynomial time, we only require her to verify an answer in polynomial time. ("My boss says I should go to Albuquerque, then Boston, then Chicago, then Denver. Is that really the fastest route?" "Hang on. I'll get back to you in n^k seconds.") If a problem is of type NP, then we can find the right answer using a "brute force," guess-and-check strategy. Unfortunately, as the number of moving parts increases, the number of possible answers explodes, and the the time required to solve the problem can get a lot longer.

Finding the solution to NP problems when n is large can take a ridiculously long time. Some NP problems are so computationally intensive that you could take all the computers in the world and multiply their processing power by a billion, run them in parallel for the lifespan of the universe, and still not find the solution. Problems or this type are perfectly solvable, but too complicated to be solved perfectly.

There's a third class of problems, called "NP-hard," that are even worse. Things are bad enough already that I'm not going to get into them here. You may also hear about "NP-complete" problems. These are problems that are both NP and NP-hard. (See this very helpful Venn diagram from wikipedia.)

The P !=NP Conjecture
But there's an intriguing catch. Until last week, nobody was sure whether any truly NP problems existed.** Maybe, just maybe, some mathematical or computational trick would allow us to convert tricky NP problems into much easier P problems.

Most computer scientists didn't believe that this was possible -- but they weren't sure. Thus, P!=NP was a longstanding conjecture in the field. (By the way, "!=" means "does not equal" in many programming languages). Many proofs had been attempted to show that P=NP, or P!=NP, but all of them had failed. In 2000, the Clay Mathematics Institute posed P!=NP as one of seven of the most important unsolved problems in mathematics. They offered a million dollar prize for a proof or refutation of the conjecture. The prize has gone unclaimed for a decade.

Last week, Vinay Deolalikar (cautiously) began circulating a 100-page paper claiming to prove that P!=NP. His proof hasn't been published or confirmed yet, but if he's right, he's solved one of the great open problems in mathematics.

So what?
Let's assume for the moment that Deolalikar has dotted his i's and crossed his t's. What would a P!=NP proof mean?

One immediate implication is in code-breaking. Public key encryption -- the basis for most online banking, e-commerce, and secure file transfers -- relies on the assumption that certain mathematical problems are more than P-difficult. Until last week, that was just an assumption. There was always the possibility that someone would crack the P-NP problem and be able to break these codes easily. A P!=NP proof would eliminate that threat. (Other threats to security could still come from number theory or quantum computing.) For wikileaks, credit card companies, and video pirates, this is reason to celebrate.

On the downside, a proof would put a computationally cheap solution to the traveling salesman problem (and every other NP-but-not-P problem) permanently out of reach. This includes a host of practical problems in logistics, engineering, supply chain management, communications, and network theory. Would-be optimizers in those areas will sigh, accept some slack in their systems, and continue looking for "good enough" solutions and heuristics. Expect a little more attention to clever approximations, parallel processing, and statistical approaches to problem solving.

A P!=NP proof can also help explain the messiness and confusion of politics and history. Recent work in game theory demonstrates that finding stable solutions in strategic interactions with multiple players (e.g. solving a game) is a computationally difficult problem. If it turns out that reaching equilibrium is NP, that would put bounds on the quality of likely policy solutions. For instance, we can pose the problem of writing legislation and building a coalition to ratify it as an optimization problem over policy preferences, constrained by preferences. If finding an optimal solution takes more than polynomial time, all the polling, deliberation, and goodwill in the world might still not be enough to arrive at a perfect (i.e. Nash, and/or Pareto-optimal) solution. In the meantime, politicians will make mistakes, politics will stay messy, and history will stay unpredictable.

Finally, a P!=NP proof puts us in an interesting place philosophically, especially when it comes to understanding intelligence. As with engineering, optimal solutions to many problems will always out of reach for artificial intelligence. Unless computation becomes much cheaper, a computer will never play the perfect game of chess, let alone the perfect game of go. Other difficult (and perhaps more useful) problems will also stay out of reach.

For those who take a materialistic view of human intelligence -- that is, your mind/consciousness operates within a brain and body composed of matter -- a P!=NP proof puts the same constraint on our intelligence. As fancy and impressive as our neural hardware is, it's bound by the laws of physics and therefore the laws of algorithmic computation. If that's true, then there are difficult problems that no amount of information, expertise, or intuition will ever solve.

The jury is still out on Deolalikar's paper. But if he's right, this isn't guesswork or extrapolation -- it's a matter of proof.

* Your friend solves logic puzzles; she's not an oracle. To give you the right answer, she needs to have all the relevant information when she begins, so it's no good asking her to pick winning horses in the Kentucky Derby, or guess how many fingers you're holding up behind your back.

** To be precise here, all P problems are also NP. If I can give you the solution in polynomial time, I can also verify the solution in polynomial time. When I say "truly NP," I mean problems that are NP but not P, such as NP-complete problems. This is the class of problems whose existence has been unclear.

Inkscape

2010-08-08T16:36:00.000-07:00

If you haven't tried Inkscape yet, you should. It's an open-source graphics program, like Illustrator, but cheap. Very functional, totally free, the built-in documentation is pretty solid. Since it's open source with a good user base, I expect development to come along rapidly. Worth a look.
Here's a link to the Inkscape showcase.

Here's a link to an essential tool in Inkscape: converting bitmaps to vector graphics.

Here's me showing off the result of an hour playing around with bitmaps, fonts, and vector paths: a spoof on the Baby Einstein logo.

"Fail small and fast"

2010-08-07T11:33:00.000-07:00

Here's an interesting interview with Google's Peter Norvig on Slate's The Wrong Stuff (found via Marginal Revolution). The topic is Google's approach to innovation, error, and failure. I really like the admonition to "fail small and fast."

Just to chip in my own two cents -- I've discovered that academic culture* is intolerant towards failure, errors, and mistakes in general. (You learn this fast when you have a lot of bad ideas.) We treat research like building a space shuttle, not looking for a decent restaurant. We do a great job critiquing ideas and tearing apart arguments, but we're often not very good about brainstorming or being willing to fail publicly. We're very reluctant to accept the null hypothesis.

Why? It's always struck me as really odd that an occupation that is ostensibly about new ideas, innovation, and pushing the status quo ends taking such a slow-moving, conservative approach. I'm still not sure why. Some working hypotheses:

Like politicians, professors' careers live and die by reputation. We can't afford to fail publicly. Those who do lose credibility and are forgotten.
Grad school selects for the risk-averse. Bigger bets and more daring would benefit the field, but those sorts of people channel themselves to jobs that pay more and don't take 5 extra years of schooling to
Committee decisions favor the cautious. Decisions about funding, tenure, and publication are all made by committee, usually anonymously.
The old guard kills innovation. If you made your name building up theory X, and some upstart comes along arguing that theory Y is better, it's only natural to resist. I don't believe the strong, cynical version of this hypothesis, but I've seen a number of examples of generational inertia.
The publication scale makes big bets risky. It takes several years to publish a book. If you have to write one book to graduate and another to get tenure, you don't have a lot of time to write failed books. Smaller increments might allow for more failure and therefore more innovation.

*This may be limited to political science; I'm not as immersed in the norms of other disciplines.

Link mishmash

2010-08-05T08:45:00.001-07:00

Four links worth a quick visit.

1. "Minimum parking requirements act like a fertility drug for cars." Interesting NYTimes editorial on the consequences of government-mandated parking. I'm interesting in getting the libertarian take on this one. Do minimum parking requirements distort the market for land use by overproducing parking lots and roads, or do they facilitate commerce by lower transaction costs? Hat tip: David Smith.

2. An infographic from Bloomberg Businessweek on negative buzz, doping, and Lance Armstrong's reputation. The interesting thing here is the source: automated buzz tracking services applying sentiment analysis to the blogosphere. This is a technology I plan to use in my dissertation.

3. The myth of a conservative corporate America - A nice infographic on corporate donations to politics. Good use of FEC data.

4. Some good resources for screen scraping

"Information is good"

2010-08-05T08:43:00.000-07:00

Picking up a thread from the monkey cage (great blog--worth a look), Slate has an article by Washington Post reporter Anne Applebaum arguing that "WikiLeaks' data-dump reporting simply makes a case for the existence of the mainstream media."

The article is a quick read. I want to pick it apart, and see where it takes us.

Things Applebaum gets:

Facts need interpretation to be useful.
In this instance WikiLeaks provided lots of facts and very little interpretation.

Things Applebaum misses:

"Mainstream media" is not the only institution that can provide useful interpretation, context, expertise, etc.
Transparency (making data public) breaks up mainstream media's monopoly on interpretation by allowing others (e.g. bloggers) to tell different stories.
Transparency allows savvy readers to come to their own conclusions, instead of relying on reporters to decide which facts are important. Most people won't bother to do this. But it's useful to have multiple perspectives.

My takeaways:

Data dumps don't replace mainstream media -- but they do threaten its monopoly on interpretation. The slightly defensive tone of the article suggests that Applebaum agrees, whether or not she wants to admit it. Don't get me wrong. I'm not a Big Media hater, but I am a big fan of transparent reporting.
On a more philosophical level, the difference between raw data and interpretation leads to some interesting ideas about "information." Raw data is not actionable. Is it still information? For most people, the dense, jargon-laden text of military reports don't mean anything. Reading them doesn't give you any traction to change your opinion or do anything differently -- they are "uninformative." A few experts (e.g. troops in Afghanistan, military experts) know how to unpack and sift through these reports. Apparently, what constitutes "information" is in the eye of the beholder.

Mutiny, information technology, and technocracy

2010-07-23T06:07:00.000-07:00

Responding to David Brooks, on the recent upsurge in technocracy and its risks. (This post was originally part of a discussion thread with friends, but I got into it enough to decide to put it up here.)

I'm convinced technology is increasing the marginal truthfulness of many progressive claims. I don't believe it fundamentally changes the relationship among individuals, economies, government, and other social institutions.

The example that comes to mind is mutiny. Mutiny was a huge worry for captains and navies for hundreds of years, right up until the invention of the radio. See mutiny for interesting reading, and watch Mutiny on the Bounty and The Caine Mutiny (starring Humprey Bogart!) for good fictional accounts of the psychology and institutions that shape mutiny.

Radio was a game changer. Since the invention and adoption of radio, mutiny has been almost unheard of, especially among large naval powers. (The Vietnam-era SS_Columbia_Eagle_incident is the exception that proves the rule.) Shipboard radios tighten the link between the captain's authority and the worldwide chain of command. It makes escape extremely unlikely for mutineers. However, desertion, disobedience, cowardice, incompetence, corruption, theft, etc. are still problems for ships and navies.

As I see it, mutiny was already a marginal activity -- very risky for the mutineers, with a low probability of success -- and radio pushed the marginal success rate much lower. But mutiny is just one act. From the perspective of naval efficiency, radio changed the balance of power, but didn't fix the underlying social problems of enforcing discipline and coordinating action. Radio caused changes in the social structure of ships, but they didn't fundamentally alter the problems that navies face.

Information technology is doing a similar thing today. It lowers the cost of storing, transmitting, aggregating, and manipulating data. Where lower transaction costs can solve social problems, the progressives (and I'm one of them, cautiously) are right to be optimistic.

But many kinds of information have been cheap for a long time. Socially, we haven't solved the problems of greed, lying, bureaucratic turf wars, bullying, corruption, graft, incompetence... When *those* are the real causes, changes in information technology can't be expected to help nearly as much. We need to invent better institutions first.The example that comes to mind is mutiny. Mutiny was a huge worry for captains and navies for hundreds of years, right up until the invention of the radio. See mutiny for interesting reading, and watch Mutiny on the Bounty and The Caine Mutiny (starring Humprey Bogart!) for good fictional accounts of the psychology and institutions that shape mutiny.Radio was a game changer. Since the invention and adoption of radio, mutiny has been almost unheard of, especially among large naval powers. (The Vietnam-era SS_Columbia_Eagle_incident is the exception that proves the rule.) Shipboard radios tighten the link between the captain's authority and the worldwide chain of command. It makes escape extremely unlikely for mutineers. However, desertion, disobedience, cowardice, incompetence, corruption, theft, etc. are still problems for ships and navies.As I see it, mutiny was already a marginal activity -- very risky for the mutineers, with a low probability of success -- and radio pushed the marginal success rate much lower. But mutiny is just one act. From the perspective of naval efficiency, radio changed the balance of power, but didn't fix the underlying social problems of enforcing discipline and coordinating action. Radio caused changes in the social structure of ships, but they didn't fundamentally alter the problems that navies face.Information technology is doing a similar thing today. It lowers the cost of storing, transmitting, aggregating, and manipulating data. Where lower transaction costs can solve social problems, the progressives (and I'm one of them, cautiously) are right to be optimistic.But many kinds of information have been cheap for a long time. Socially, we haven't solved the problems of greed, lying, bureaucratic turf wars, bullying, corruption, graft, incompetence... When *those* are the real causes, changes in information technology can't be expected to help nearly as much. We need to invent better institutions first.">

Top secret America: Simultaneously sinister and incompetent

2010-07-19T06:16:00.000-07:00

The Washington Post just put out an investigative report on "Top Secret America" -- federal departments and agencies with top secret security clearance.

Here's the trailer. Yes, a 1:47 Flash video trailer for a report put out by a newspaper. It has a Bourne Ultimatum feel to it, right down to the percussion-heavy, digital soundtrack and gritty urban imagery. Anybody want to talk about the blurring lines between news and entertainment?

WPost claims that a fourth, secret branch of government has opened up since 9-11. They make this new branch out to be simultaneously sinister and incompetent (like Vogons). Watch the trailer and see what they're hinting at.

Some counterclaims: government is still learning how to make use of recent (i.e. within the last 30 years) advances in information technology. (Take this article in the Economist on the alleged obsolescence of the census, and data.gov as examples).

Since the technology is new, it makes sense that new agencies would develop to handle the load.
Information technology is useful, and the government has a lot of data to process, so it makes sense that there would be a lot of people involved.
Since business tends to be more nimble about technology adoption, it makes sense that a lot of the work would be outsourced to private firms, at least initially.
Since the process is unfamiliar, it makes sense that there would be some inefficiency.
Since we want the system to be robust to failure, it makes sense that there would be some redundancy.

I agree with the Post that "we want to get this right," but let's back off the hype and the alarmism a little bit.

a jaundiced formula for spinning educational research into something that sounds interesting

2010-07-13T07:49:00.000-07:00

Here's a jaundiced formula for spinning educational research into something that sounds interesting. Most researchers and reporters seem to follow this formula pretty closely*.

1. Sample a bunch of kids in category A, and a bunch of kids in category B.

Ex: A kids have computers in the home; B kids don't
Ex: A kids are white; B kids are nonwhite
Ex: A kids go to charter schools; B kids don't

2. For each group, measure some dependent variable, Y, that we care about.

Ex: grades, SAT scores, dropout rates, college attendance, college completion, long term impacts on wages, quality of life, etc.

3. Compare Y means for group A and group B.
3a. If the means differ and the A versus B debate is contested, take a side with the group A.
3b. If the means don't differ and many people support one option, take the opposite stance. (Ex: "Charter schools don't outperform non-charter schools")
3c. If neither of those options works, continue on to step 4.

4. Introduce a demographic variable X, (probably gender or SES) as a control or interaction term in your regression analysis. It will probably be significant. Claim that A or B is "widening the racial achievement gap," or "narrowing the gender gap," etc. as appropriate.

Papers following this formula will frequently be publishable and newsworthy. (You can verify this, case by case, with the studies cited in that NYTimes article.) They will rarely make a substantive contribution to the science and policy of education. Awful. Awful. Awful.

Why? Because this approach is superficial. The scientific method is supposed to help us understand root causes, with an eye to making people better off. But that depends on starting with categorizations that are meaningfully tied to causal pathways. The distinctions we make have to matter.

In a great many educational studies, the categories used to split kids are cheap and easy to observe. Therefore, they make for easy studies and quick stereotypes. They feed political conflict about how to divide pies. But they don't matter in any deep, structural way.

Example: Does having a computer in the house makes a kid smarter or dumber? It depends on how the computer is used. If the computer is in the attic, wrapped in plastic, the effect of computer ownership on grades, SAT scores, or whatever will be pretty close to zero. If the computer is only used to play games, the effect probably won't be positive; and if games crowd out homework, the effect will be negative. No real surprises there. And that's about as far as these studies usually go. "Computers not a magic bullet. Next!"

This is more or less the state of knowledge with respect to school funding, busing, charter schools, etc. We know that one blunt policy intervention after another does not work miracles. We haven't really gotten under the hood of what makes the complex social system of education work. It's like coming up with a theory of how airplanes fly based on the colors they're painted. ("White airplanes travel slower than airplanes painted camouflage colors, but tail markings have little effect on air speed.) You may be able to explain more than nothing, but you certainly haven't grasped the forces that make the system work.

To say the same thing in different words, scientists are supposed to ask "why?" Studies that say "kids in group A are more Y than kids in group B" doesn't answer the why question. They are descriptive, not causal. Without a deeper causal understanding of why schools work or don't work, I don't think we're ever going to stop chasing fads and really make things better.

*This is an epistemological critique of just about every quantitative article on education. In general, I'm supportive of the increasing influence of economic/econometric analysis in education policy, but this is one area where we quants may be making things worse, not better. Hat tip to Matt for sending the article that reminded me how much the failings of this literature frustrate me.

Google code -- pros and cons?

2010-07-02T05:56:00.000-07:00

Anybody have thoughts/experience on the pros and cons of Google code?

Reliability

2010-07-01T08:04:00.000-07:00

I'm sold on the argument that Krippendorff's alpha is the way to go for most reliability calculation. Unfortunately, it's been hard to find code to calculate it. Here are some of the best links I've run across.

The NLTK package for python has code for computing alpha. It looks like this does basic nominal calculation; I don't know if/how it copes with missing data.

The concord package in R does nominal, ordinal, interval, and ratio versions of alpha. It looks like this might not be maintained anymore, but it works.

Here's a nice page of resources by computational linguists Artstein and Poesio. Unfortunately, what they show is mainly that there aren't very good resources out there. Their review article is very good -- the best treatment of reliability I've seen in the NLP community so far.

Deen Freelon has some links to reliability calculators and resources, including two nice online reliability calculators: Recal-OIR, and Recal-3.

Krippendorff's oddly-formatted, information-sparse web page. He invents the best measure for calculating reliability, then keeps a lid on it. Less animated bowtie dogs, and more software, please!

Matthew Lombard has a nice page on reliability statistics and the importance of reliability in content analysis in general.

Beg: Does anybody know how to compute K's alpha for a single coder?

I have data coded by several coders and need to know who's doing a good job, reliability-wise, and who's not. At a pinch, I'd be willing to use a different reliability statistic, or even an IRT model. It just needs to be statistically defensible and reasonably easy to code.

How to catch a dirty pollster

2010-06-29T13:24:00.000-07:00

Just a couple hours ago, Markos Moulitsas (you know, the Daily Kos) announced that he's suing his former polling group, Research 2000. Evidently, they faked a lot of their data in the last 19 months, and didn't even have the decency to fake it well.

The statistical case against R2000 is made here. Nate Silver chimes in here.

The punchline:

"We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls."

What's interesting to me about the situation is how clear-cut the evidence is. Mark Grebner, Michael Weissman, and Jonathan Weissman (GWW), conduct three tests to show that the numbers are cooked. Each statistical test gives results that are devastatingly improbable.

Probability of odd-even pairs between men and women: 776 heads on 778 tosses of a fair coin, or about 1 in 10²³¹.
Probability of consistency of results in small samples: ~1 in 10¹⁶.
Probability of missing zeroes in a time trend: ~1 in 1016.

Statistically, there's almost no wiggle room here. Until the R2000 folks come forward with some really compelling counterevidence, I'm convinced they're frauds.

An interesting feature of the last two patterns is that they hint at overeagerness to give results. Patterns in the data are *too* clear, and the detailed crosstabs supplied by R2000 made it easier to catch them. If they had been willing to dump some random noise into the equations -- blurring the results -- it would have been much harder to build a conclusive case against them.

This jives with my experience doing survey research for clients: nobody wants to pay for null results. Even when your data collection is legitimate, there's always pressure to twist and squeeze the data to wring out "findings" that aren't really there. Three cheers for transparency and the scientific method.

Reasons to study political blogging

2010-06-28T19:51:00.000-07:00

I'm working like crazy on my dissertation prospectus. Data work, lit reviews, etc. To escape from early research purgatory, I plan to blog parts of the prospectus as I write them.

I'll kickoff today with introductory definitions and motivation. Feedback is much appreciated. Beware of dry, academic writing!

What is a blog?
Paraphrasing wikipedia, a blog is a website containing regular entries ("posts") of commentary, links, or other material such as photos or video. On most blogs, posts are displayed in reverse-chronological order -- the most recent post appears first. Although most blogs are maintained by individuals, some are run by small groups, and blogs speaking on behalf of corporations, churches, newspapers, political campaigns, etc. are increasingly common. Many blogs focus on a specific topic, ranging from broad to narrow: entertainment, cooking, astronomy,the Detroit Tigers, to cold fusion. For my dissertation, I plan to focus on political blogs.

Why study political blogs?
Here are five reasons to study political blogs.

Blogs are public facing. Lots of people read them, including politicians and journalists. The extent to which blogs are replacing mainstream media is an open question, but it's certain that blogs have come to play an important role in public discourse, with real impact on politics.
Bloggers span a wide variety of opinions. The blogosphere embraces everyone from conservative wingnuts to liberal moonbats to political moderates. Some political bloggers are politically omnivorous, writing about anything political. Others focus on specific issues and topics: foreign policy, Congress, feminism, etc.
Bloggers include both experts and amateurs. Dividing the same pie in a different direction, many A-list bloggers (e.g. Andrew Sullivan, Ariana Huffington, Glenn Reynolds, Michelle Malkin) clearly qualify as political elites: they are experts, immersed in politics, well-informed and well-connected. Other political bloggers are more obscure, casual -- closer to the average Joes who make up the "mass public."
Blogs are updated frequently. This has two nice consequences. First, frequent posts allow us to replay bloggers' reactions to events as they unfold. Second, frequent posts mean we have a lot of posts to work with.
Blogs are archived publicly. Unlike most forms of political speech and action, blogging leaves a permanent data trail.

The combination of these attributes creates a kind of perfect storm for social science. Understanding the flow of opinions and ideas has always been difficult for social scientists, because most of our data have come from surveys.