Power Laws of Information (Draft)

Forbes has an interesting article this month that explores the vast quantity of information our society produces (furled). In 2002 it is estimated that the world created about 5 exabytes of data. For comparison, one exabyte is what a digitized copy of all words ever spoken by human beings would take to store–in one year, our machines created five times that amount of information (5,000 times the amount of all printed material in the world).

Meanwhile, Kottke notices that people can form groups far more numerous than themselves, something he calls “The 2 to the Nth feature” (actually he called it “The N-squared feature” before someone corrected his math). He notes that in music, with N songs, you can create 2^N playlists; blogs with N posts can have 2^N “categories”; and with N news feeds, 2^N feed collections.

Because of this digital leverage, the ability to create much more metadata than original information, we have vastly more data on our hands than we can handle in conventional ways. The focus of work, then, will almost certainly shift from trying to understand all the information to trying to collect it in managable groups.

It’s what we did with our latest design at National, which allows users to compile their own collections from our vast database of financial metrics. Steven Johnson calls it “curatorial culture“, my freshman year dormmate (and inventor of “googlebombing”) Adam Mathes is studying the suddenly-hip field of library science, Apple’s wildly successful “Celebrity Playlists” (cited by Johnson) just added “Party Mixes“, which allow you to rearrange your collections on the fly, it’s part of the Experience Economy (putting your trust in a person/group whose collected beliefs you most identify with), it’s an advanced interface concept (filtering by common attribute — 37signals SVN discussion).

Some people lament the fact that when a large company takes the lead in creating these collections, they also lock the customer in to their company’s specific solution. In Apple’s case, the celebrity playlists all provide a one-click link to buy the songs from Apple, which triggers a download of Apple’s proprietary data and encryption format. Compare that with receiving a mix tape or CD from a friend, which plays a universal format and can be stored separately from other devices (to be fair, Apple has chosen AAC, an open standard, and offered to license their “FairPlay” encryption to others; more than Microsoft has ever done with their formats).

But with so much data flooding our lives, those concerned with resisting the proprietary formats it takes are missing the bigger wave of the collections that are the real product. Apple isn’t selling files, it’s selling experiences. The format those experiences are delivered in will certainly change over time; but since the product is an experience–limited by time–its “obsolecence” or “locked-in format” is meaningless. I don’t care if something is AAC, MP3, HTML, CSS, paper, LCD, written, shouted, or telepathed–as long as I have the experience of receiving it, it has become part of my social and cultural experience. As I’ve said before, people are the ultimate futureproof technology, especially those in cultures and societies with others.

This data explosion is not something we are meant to store, capture, and ingest–it is something we are meant to experience. Let go of it, ride the wave of data, and learn how to stop worrying and love the information bomb.