Nielsen and foul-weather Luddites

This is a story I’ve been meaning to post on for several weeks now, but as previously mentioned, time has kind of been at a premium. Like the marketing and statistics nerd I am, I thought this story on the rising tide of opposition to Nielsen’s newest system of obtaining television ratings was an interesting one.

Most have heard of Nielsen, the company responsible for giving us the television ratings that dictate which shows get picked up again, which get the axe, and when sweeps are, all in service to networks who need proof of some kind of audience so they can charge advertisers more for airtime. Nielsen’s primary method for collecting this information locally has been via paper diaries: a representative sample of our nation’s population keeps a daily record of what they watched and when. When I say local, I mean in the top ten television markets: LA, New York, Chicago (the primary cities where changing this method has become an issue), Boston, and other large cities.

It stands to reason that, if you ask people to keep a handwritten diary of what they watched on TV, it’s not going to be very accurate, particularly beyond the first couple days or so. So Nielsen devised a new method of collecting this information called a People Meter, where a device monitors what channel the television is on and who is watching it (recorded by having the people present press a button to tell it they’re present--admittedly still far from perfect, but a definite improvement over paper diaries). This device is already used to collect national ratings information, but a number of people have protested the implementation on the local level because in the aforementioned cities, populations are far more diverse, and minorities living in those cities want that diversity to be captured with some accuracy. That accuracy is in question with the People Meters because the results from them in test runs are very different from paper diary results, most notably in ratings of shows geared towards minorities. This has led to claims that Nielsen is undercounting minorities in these cities, as well as lawsuits and calls to delay full scale usage of the People Meters. The threatened results of said undercounting: cancellation of TV shows popular chiefly among minorities, leading to fewer jobs for minorities in entertainment and a diminished presence in popular culture.

This issue makes me mad for so many reasons; actually, for one reason--that it’s ridiculous--and there are lots of reasons why. First of all, the only reason this is getting any attention in the first place is because News Corp., which owns Fox, backed the opposition with a couple million dollars. Do you think some of Fox’s shows were adversely affected by the new system of collecting data? Yes. Across the board (not just on Fox), shows thought to be popular with minorities saw a sharp drop in viewership when using People Meters as opposed to the paper diaries, which is what prompted the undercounting claim. Which of these methods, then, is at fault? Common sense would argue that paper diaries are far more given to error, particularly when it comes to minorities. They are historically less likely to fill them out, and for this reason, Nielsen oversamples minorities. In general, people are diligent for a few days, then lapse and fill them out at the end of the week (or whenever Nielsen comes to pick them up) from recall. Right there you lose the data from channel surfing. In addition, paper diaries give people the chance to be dishonest about what they watched, and I think this loophole is the culprit here. Shows aimed at minorities may be getting higher ratings than they should because they want to show support and so they say they watched these shows. That option isn’t available with People Meters. I doubt the minority families chosen by Nielsen are conspiring to up the ratings of shows they think they should be watching, but we aren’t talking huge samples here (of any group), and it doesn’t take much to change a show’s rating when you consider the fact that only a few percent of the population is really watching it anyway.

It makes me angry that these groups are actually impeding progress. Nielsen’s new system is such a clear improvement over the old one, and sampling problem claims aimed at an organization that does this for a living are just plain silly. I’m not a huge proponent of Nielsen or television in general (it is a wasteland, after all), but someone has to do what they do, and I support accuracy in collecting data, particularly if it shows how network programming sucks more each year (I mean did we really need a second iteration of the Simple Life?).

Update: Another blog's more in-depth coverage of this issue (I knew there had to be other marketing dorks out there).