Facibus Reviews

A recent posting in SIGIA-L referred to a NY Times article on a Columbia University study. The article by Duncan Watts discusses how popularity affects choice. In Duncan’s own words:

Recently, my collaborators, Matthew Salganik and Peter Dodds, and I conducted just such a Web-based experiment. In our study, published last year in Science, more than 14,000 participants registered at our Web site, Music Lab (www.musiclab.columbia.edu), and were asked to listen to, rate and, if they chose, download songs by bands they had never heard of. Some of the participants saw only the names of the songs and bands, while others also saw how many times the songs had been downloaded by previous participants. This second group — in what we called the “social influence� condition — was further split into eight parallel “worlds� such that participants could see the prior downloads of people only in their own world.

And the result? Again, in Duncan’s own words:

So does a listener’s own independent reaction to a song count for anything? In fact, intrinsic “quality,� which we measured in terms of a song’s popularity in the independent condition, did help to explain success in the social-influence condition. When we added up downloads across all eight social-influence worlds, “good� songs had higher market share, on average, than “bad� ones. But the impact of a listener’s own reactions is easily overwhelmed by his or her reactions to others. The song “Lockdown,� by 52metro, for example, ranked 26th out of 48 in quality; yet it was the No. 1 song in one social-influence world, and 40th in another. Overall, a song in the Top 5 in terms of quality had only a 50 percent chance of finishing in the Top 5 of success.

In our artificial market, therefore, social influence played as large a role in determining the market share of successful songs as differences in quality. It’s a simple result to state, but it has a surprisingly deep consequence. Because the long-run success of a song depends so sensitively on the decisions of a few early-arriving individuals, whose choices are subsequently amplified and eventually locked in by the cumulative-advantage process, and because the particular individuals who play this important role are chosen randomly and may make different decisions from one moment to the next, the resulting unpredictability is inherent to the nature of the market. It cannot be eliminated either by accumulating more information — about people or songs — or by developing fancier prediction algorithms, any more than you can repeatedly roll sixes no matter how carefully you try to throw the die.

So… social influence is as important as quality - the more popular something gets, the more popular it is likely to get. This phenomenon is sometimes referred to as Cumulative Advantage.

There is a popular concept in Information Architecture - the “Best Bet” search result. This is the one page that you want at the top of all searches on a given term. Microsoft uses them on their intranet - and some of their scientists are looking at how they may be detected automatically. It’s the equivalent of defining the “I’m Feeling Lucky” button in Google search.

Duncan’s study (and the “bad influence” that social influence has on defining popularity) has been used to suggest that Best Bet searches are not as useful as they could be - that the most popular site/page/blog posting is not necessarily the best. Interesting. Anyone over the age of 20 who has listened to a top 40 radio station has probably wondered how some of the songs are popular - it is because of the “herd behaviour” of popularity.

I’d have to add that just because popularity may be independant (to some extent) of quality, it does not follow that a thing is of poor quality just because it is popular - sometimes the “wisdom of the crowd” gets it right.

Microsoft’s experience on their own intranet was a good case for Best Bets. They may have had, say, 10000 pages on Excel 97 across the whole intranet. The Best Bet page was, from what I understand, allocated to the product group responsible for maintaining Excel 97 - the folks who were the definitive source of information on what could or could not be done across the whole organisation in the specific instance of Excel 97. Everyone else’s macro tips pages and marketing war stories were not disabled, they were just placed one step lower in the search results. I think that this is a good use of the Best Bet concept.

That the Best Bet concept has been abused is beyond dispute - but I do not think that it is totally busted.