David Weinberger‘s new book “Too Big To Know” (#2B2K – be sure to pick book titles that make good hash tags…) launched last night at Harvard Law School with a talk entitled “Unsettling Knowledge”. If you know David’s work, it’s obvious that the title is a pun. And David’s new book is a wonderfully unsettling piece – it challenges our notion of what knowledge is, and introduces the uncomfortable question of how we navigate this new space.

Knowledge as we know it is coming apart, David tells us. The bastions of knowledge, the physical emblems of knowledge, like encyclopedias, newspapers and libraries are undergoing radical transformation. We know we’re heading into a future that’s deeply different, though we don’t know quite how. The manifestations of knowledge are at risk, and all it took was the touch of a hyperlink.

How did these institutions fall apart so quickly? It’s an impossible question to answer, but he offers one path through the thicket. He starts with a famous quote from Daniel Patrick Moynihan, who tells us “Everyone is entitled to his own opinion, not his own facts.” This is the promise of knowledge: that if we all got together and had an honest conversation, we can eventually come to an agreement. There is knowledge and it can bring us together.

We tend to assume that knowledge gives us an accurate picture of the world, built up bit by bit, fact by fact. In acquiring knowledge, we nail down each piece with certainty. And we see knowledge as a product of filtering and winnowing – we move from perception to true perception, from a mob of opinion to true belief. Knowledge is about finding gold within the flux.

We’ve always had to filter, based on the fact that the world is way bigger than what fits in our skills. There’s too much to know (quoting Anne Blair’s book “Too Much to Know“) and the world is too big to know.

Traditionally, we’ve handled this by breaking off a brain-sized chunk of the world and getting an expert to understand it. Once we’ve got that expert, we can stop asking questions: we simply ask the expert. Experts, and the credentials that create them, are stopping points. They’re points beyond which we don’t need to look any further.

But that’s how knowledge works on paper. Books, for all their magnificence, are a disconnected medium. They are contained within covers, they are shelved apart, they don’t naturally connect to one another. The author’s job is to put everything she knows on a topic between two covers. The arguments move in sequence, from the beginning to the conclusion. And because the book is an essentially limited medium, good writers ruthlessly cast things aside, deciding what it put in the book and what is excluded. Books are born of long-form arguments, moving us forward step by step, brick by brick.

Links are a new form of punctuation. They give you a means of continuing. In the print world, to follow a footnote in a book, you need to get on a bus and go to the library. That’s why we don’t generally follow footnotes. But now we can jump from one book to the next. It’s a magic map – touch a place on the map and you go there.

The internet is an environment that’s all about connection and our knowledge is picking up properties of the medium. Knowledge in this space is characterized by the fact that it’s “too much, messy, unsettled, and unstructured”.

Clay Shirky suggests that there’s no such thing as information overload, only filter failure. This is a very modern response to an older question. Futurist Alvin Toffler warned us about information overload, popularizing the phrase. It’s an extension of the idea of sensory overload, the idea that too much input could overwhelm and paralyze you. This is based on the faulty assumption that brains are information processing machines, and that we can overwhelm and crash them.

This line of thinking led marketers to conclude that choosing between 16 brands would be overwhelming to an American housewife and that fewer choices needed to be offered. But we’re now headed to a point where there’s an exabyte of genomic information available, and that number doesn’t lead us to paralysis, but to fascination. We’ve redefined the term “information overload” through how we use it.

We’re less overwhelmed because we’re learning different ways to filter. When we filtered in the print world, we did so in a way that prevented us from seeing the dregs. We saw only the books that our local library chose to buy, and only the books the publisher chose to print. The manuscripts filtered out of that process were invisible to retrieve through ordinary means.

Now, in a digital age, we filter forward, not filter out. All that information – some of it very low quality – is out there somewhere on the internet. We could curate and try to delete the stuff that’s wrong, hurtful, harmful or hateful. But it’s expensive to exclude information and cheaper to include everything. When you curate, you’re making decisions about what is interesting to your users, and no one can accurately predict what might be useful to a researcher in the future. Filter out all the gossip and crap from new media and you harm the scholar who wants to study celebrity behavior. You couldn’t have predicted the high level of interest in notes from a committee meeting in Wasilla, Alaska in 2008 until Sara Palin became a public figure.

The web has worked by developing tools that include all content and filter when we retrieve it. As recently as a decade ago, information retrieval experts told us that ordinary users would never use tools this complicated. But now we use them everyday, because we have to. And we’re seeing much better tools, like Shelflife, the tool Harvard’s Library Lab has created to allow users to browse the vast set of information in Harvard’s library systems.

We don’t just have a lot of information – the information is very messy. We like order – David shows a slide of zoological specimens, beetles mounted on pins – and we’re very good at establishing it. We understand where everything fits in a tree of species, based on similarities and differences. To know where a species fit into this tree was to know how the world works – to not know it was to be adrift.

In the physical world, there’s only one way to sort manifestations of information. You might want to sort your CDs by artist, while your partner might want them sorted by genre. There’s only one possible they can be stacked on the shelf, because no two things can be in the same place at the same time. In a digital age, we simply make playlists. We end up with a mess of information, but it’s a rich and fertile mess.

Figuring out where things fit in the natural order of things was an essential piece of being human. Human beings saw ourselves as “the knowers. But there’s multiple orders and multiple ways of categorizing, through tags, playlists and other ways to sort information. Messiness is an essential feature of how we scale meaning. But, David warns, we still tend to think of knowledge in the ways we did when books had to sit on a single place on the shelf, when knowledge had a single, possible, right form, rather than multiple forms.

Knowledge is too big, messy and wildly unsettled, just like the internet. “For every fact on the internet, there is an equal and opposite fact.” David warns that there is nothing we all agree on – you can find someone willing to argue that 2+2 is not 4 (and, indeed, a quick Google search shows this to be true.) We don’t agree about anything, and David warns, we never will. “This doesn’t mean there are no facts – but it does mean that people are going to insist on being wrong.”

What this persistence of disagreement means is that the promise of knowledge Moynihan offers – that we can agree on a set of facts and then argue our opinions – is not going to be fulfilled. As it turns out, we don’t even know whether Moynihan said “everyone is entitled to his own opinion, not his own facts” or whether that’s exactly what he said.

The good news is that we’re rapidly developing ways of dealing with difference and disagreement. YouTube has a crummy commenting system, as is well documented and well established. David shows us a threat of comments on a recent Batman movie trailer. Somewhere deep in this comment thread is an impassioned argument about circumcision. It would have been great if YouTube supported forking of conversations. Forking is a powerful way to deal with disagreement. It’s very hard to do in the real world without social consequences – if we decide to move away from the dinner party to our own table where we talk about circumcision, it makes people uncomfortable – but it’s very easy to do this on the web.

In the 19th century, it was very challenging to classify the platypus. There was one space in a taxonomy for warm-blooded animals, and another for animals that produce eggs. Scientists thought the platypus must be a hoax, because it didn’t fit within existing categories. Even when presented with a specimen from Tasmania with eggs intact, they fought the platypus “hoax” as something that didn’t work within existing categories.

Now we can solve problems of overly rigid taxonomies by using linked namespaces. We can create a database of names, and a database of taxonomies. We can deal with the platypus and the water mole, and map scientific and colloquial names onto different possible structures. “Pick your name, pick your taxonomy and get on with your life. So what if we disagree? Yay for difference!”

David is actually quite concerned about difference, and just how much difference we can tolerate and still interact and function. He acknowledges that there’s a human tendency towards homophily, flocking together in groups united by race, gender, belief, socioeconomic status, etc. This can lead to a serious challenge to public discourse – echo chambers that can solidify beliefs, making them more extreme and polarized. But David worries that posing issues this way relies on an unquestioned assumption: that conversations are between people who disagree deeply and looking for solutions and common ground by trying to get to the facts. This analysis misses the social role of conversation. We need so much context and so much agreement to even have a conversation. “To have a good conversation, you need to have 99% similarity and 1% difference.” He suggests that some of the work Yochai Benkler and I have been doing may help us find productive paths towards including difference, but reminds us that the high level of disagreement and the difficulty of finding common ground is likely a core feature of the internet and knowledge in an internet age.

Finally, knowledge in this new paradigm is unstructured. We’re used to the idea that knowledge has a basic structure. We have grown used to long form arguments that take us from A to Z, and we’re particularly fond of arguments that take us from A to Z in an orderly path, where Z is an unexpected place to end up. “This is a magnificent form of thought, but the long form argument is losing it’s preeminence.”

We might think of Darwin as a leading proponent of the long form argument. And his argument certainly led somewhere unfamiliar. But he wouldn’t have analyzed data for years and released a massive book if he were working today. He would publish online. And even if he didn’t, the conversation about his work would be based online. Whether or not we imagine Darwin tweeting from The Beagle, the web is where the thinking about and reacting to Darwin’s work would take place, and collectively, it will have more value that Darwin’s long form work taken alone. Moving forward, we will not just see these long form works, but the webs that precede and follow them.

Michael Nielsen has recently written about scholarly community reaction to results at CERN that offer evidence for faster than light neutrinos. As these results came in, they were posted to arXiv.org, a journal preprint site. They stirred up a firestorm of interest and reactions. Some of those reactions are brilliant, some are stupid and wrong. But that welter of discussion is where knowledge is – it’s taking place outside of printed peer review journals.

Darwin spent seven years studying and dissecting barnacles before working on The Origin of Species. His two volume work on barnacles includes countless facts, and his hard work to discover and pin them down was an act of nobility. But science doesn’t work quite like that anymore. We work with clouds of data about genetics, astronomy, and other topics. These data clouds are fundamentally different than facts. When data.gov released sets of government information, they didn’t clean or normalize it ahead of time – they released raw data. They concluded that it was better to put the data out there than to constrain themselves to information that was consistent and known, for the simple reason that this constraint would have slowed them down badly. Darwin would not have agreed – he spent seven years on one fact.

There’s value in getting the data out quickly, David argues. It may be the one approach that’s scaleable – releasing raw data and letting individuals and groups clean, analyze and share what they find. Peer review scientific journals don’t scale, but perhaps peer to peer peer review might. We’re seeing growth in the Open Access journal field, particularly in spaces of repository where data is released, not peer reviewed.

One way we can start making sense of these new data sets is through the magic of linked data, a format suggested by Tim Berners-Lee, father of the web. We organize information in triples:

the platypus | lives in | Tasmania
Watermoles | lay | eggs

When we link triples to a central reference, we can resolve our platipae to water moles and link our triples together. Facts, which used to look like bricks, now look like links.

David closes by returning to his original question: why were old knowledge systems so fragile? These systems assumed knowledge was bounded, settled, orderly and proceeded step by step. But that’s not what knowledge feels like in the age of the internet. It feels unbounded, overwhelming, unsettled, messy, linked and governed by our interests. And those properties are the properties of what it means to be human in the world.

“Networked knowledge may or may not be truer about the world, but is is truer about knowing… This crazy approach to knowledge feels familiar to us, because it’s how we tend to know.” He closes with an observation that’s both hopeful and unsettling: “What we have in common is a shared world about which we disagree, not a common knowledge we share and can collectively come to.”

I’ve followed David’s work for a long time, and had the pleasure of watching him work through the ideas behind this book – David and I are both part of a group at Berkman that helps colleagues explore book-length projects. While I’m familiar with this line of David’s though, it was exciting and unsettling to hear him work through these ideas covering the whole arc of the book. I think this may be the most unsettling and radical book David’s put forth. On the one hand, it’s not a surprise that people will disagree on any concievable fact. But David’s suggestion that we give up on achieving an impossible consensus and proceed with the hard work of getting on with our lives strikes me as challenging and liberating, a very different path than I hear from most activists and advocates. I’m enjoying wrestling with the ideas David puts forth both in this talk and in the paper and hope lots of readers will take up the challenge as well.

Some notes from day 2 of the Microsoft Research Social Media Symposium:

My attempts to transcribe Wael Abbas’s talk about media and protest in Egypt prior to the Arab Spring.

Becky Hurwitz has been active in the Occupy movement in New York City, and offered reflections on how Occupy is developing and testing technology for protest. She invites us to use the people’s mic, a technology created to ensure that participants in Occupy General Assemblies can hear speakers, despite police bans on amplification. We dutifully echo her in a mic check and in repeating a few words of her talk, one three word phrase at a time.

The people’s mic is effective and accessible – anybody can use it, regardless of whether you have technical expertise or money. It’s a rhetorical leveler – when you need to speak in three word phrases, you can’t dazzle people with complex arguments. It’s an empathic technology, she argues: you are asked to repeat things even when you disagree with them. And it allows processes to be collectively enforced – people who speak out of turn will cause the mic to fizzle out, as people refuse to amplify their statements.

Attempts to use different forms of amplification to share General Assemblies were often less successful than this simple, low-tech solution. A system allowed GAs to be simulcast in Spanish, and Becky tells us that it was useful, but was rented, expensive and ended up being discontinued once funding ran out. Attempts to broadcast proceedings on low-power FM were technically successful, but required a three person team to manage mics and transmitters. Attempts to stream via smartphone failed when it became many people didn’t have smart phones, didn’t want to listen to streamed audio for hours, and disliked the latency of streaming systems.

Other technologies have experienced stumbling blocks as well. The Tech Ops group at Wall Street has tracked media streams coming out of a thousand other Occupy movements. There are at least seven aggregators of these feeds, but they’re not very popular. It’s hard to follow the conversations taking place in other movements, and other than the effect of showing how widespread a movement is, it’s unclear that an aggregator is the best way to share this data.

She closes by examining technology developed to help people when they get arrested. Several smartphone applications have been designed to provide a “panic button” which sends a pre-programmed SMS when someone triggers the alert. The idea is that a person is supposed to press the button just before they are arrested, so allies can provide legal services. These systems have been obsoleted, Becky argues, by lawyers and sharpies. Lawyers shared the phone number of the National Legal Guild, and people wrote the numbers on their hands so they could call the service when they were put under arrest. And it turns out that yelling your name and birthdate to a lawyer standing by as you are arrested is really helpful, as those lawyers can act as witnesses to your arrest and testify in hopes of getting you released.

Development of technologies in a need-based space instead of a commercial space can follow an interesting trajectory – an idea that fails to get traction dies rapidly. In a commercial setting, you’d likely iterate and try to improve on a failed project. In non-commercial space, you rarely do. Becky wonders, “how do you commit to ideas without commercial reasons to explore and refine them?” Occupy raises other interesting questions as well: “How do you create end products that users can control, like the people’s mic, where people can turn off their amplifier when people defy process? How does occupy become a loud voice on existing platforms like Facebook, Twitter and people’s email inboxes?”

Zeynep Tufekci believes we’re seeing electronic communications usher in a moment where old power and new power are coming together. This syncretic power is complex and hard to understand, but we can think of it in terms of understanding who controls the power of attention.

In the Arab Spring, which she’s been studying closely, understanding the power of attention means looking at traditional media like Al Jazeera and the New York Times, at the algorithmic power of Twitter trending topics, at networked activists and their sympathizers, at new media curators and bridge figures, and at the network power of governments and political powers.

If we want to understand attention, we need to understand focusers, people capable of transferring lots of attention. We should look at the most linked users of Weibo, the thousands of followers of Wael Abbas’s Twitter account. These are some of the building blocks of a new filtering system. We’ve needed filters to cope with the massive amount of information created from well before the Internet. While Clay Shirky’s observation that we don’t have information overload, just filter failure, is legitimate, events like the Arab Spring give us a chance to better understand what’s powering these filters and how they work.

Zeynep tells us the story of Rami Jarrar, a Syrian dissident who tweeted under the pseudonym “AlexanderPageSY”. He’s a Syrian businessman who lived in Damascus and was educated in Britain. He writes about Syria in beautiful English, and under his pseudonym, became something of a spokesman for the anti-Assad movement in Syria. Zeynep identifies him as a bridge figure, but notes that he chose to tweet only in English, translating a great deal of Arabic content into English for non-Syrian audiences.

Somehow, his real identity was discovered. He got a phone call from someone sympathetic within the Assad government who warned him to flee immediately. He left within an hour with his wife and six-month old daughter, heading first to Lebanon and then to Cairo. He was likely one step ahead of arrest or death, Zeynep tells us, given the experience of other activists arrested by the regime.

The next chapter of Rami’s story came when he tried to travel to Doha, Qatar to attend a conference. On entering the country, the border guard decided that Rami’s passport was out of order and decided to deport him. Deportation doesn’t send you back to where you’d flown in from – Cairo – but to your home country. Rami tweeted about his predicament – he was about to be deported “home” to a country where he would likely be arrested, disappeared or killed due to an inflexible bureaucracy.

A 21st century network of concerned Twitter followers emerged to address the Kafkaesque 20th century problem of bureaucracy. But the solution was ultimately a 17th century one – A Syrian living in Lebanon, who tweets as @LeShaque, knew a sheik and was able to get him up in the middle of the night to intervene in the solution. “We needed a 17th century solution – a sheik – so we used a 21st century solution to get one.” This seems ironic. “We had mobilized Al Jazeera English, Al Jazeera Arabic, media people around the world, but the only guy who could actually get him out was a sheik.” Fortunately, the network found someone with the connections and the guts to call a Qatari sheik in the middle of the night and got him to intervene.

“The 21st century network worked until the last mile, but then we needed to reach old powers.” Media activism, Zeynep speculates, is most powerful when it hacks the system and contacts the old powers. She offers the similar example of Mona El Tahawy, arrested in Egypt. She tweeted about her arrest and activated a network, but ultimately, she was released from custody (where she’d been beaten and injured) because the Twitter network reached Anne Marie Slaughter, who contacted friends in the US State Department who intervened on her behalf.

We’re seeing the emergence of micro-celebrity citizen journalist activists, like “Angry Arabiya”, know to her parents as Zainab Al-Khawaja. She’s been a brave and outspoken critic of the government in Bahrain, facing arrest and abuse. Recently, she helped stop the movement of riot police into a Manama neighborhood by appearing in an abaya (which Zeynep identifies as giving female activists an interesting new form of power, exercising the image of being conservative and observant) and blocking the procession of police vehicles. She resisted arrest, and the police weren’t initially able to arrest her, as they lacked enough female officers to detain her. Bahraini officials, Zeynep tells us, debated arresting her and concluded that they didn’t want the pictures of her being dragged off to appear in the New York Times.

Microcelebrities are learning how to exert their power, to stop the movement of troops and to help counter torture when they’re in custody. It’s far from the power to end police states, but it’s an interesting new capability and a sign of things to come.

Danny O’Brien of Committee to Protect Journalists titles his talk “Public Private Secret Alone”. He suggests that, contrary to many people’s view of lessening privacy, we may be entering a golden age of privacy. We’re starting to codify what should be public and what should be private, what’s shareable and what’s sacrosanct. This is at least as much a social process as a technical one, and Danny thinks that we’re all now learning to use the right “register” in speaking in online spaces.

Putting his tongue firmly in cheek, Danny declares that he’s likely the first person to have thought of the distinctions between personal and public speech, blogging on the topic in 2004. “As a blogger, I looked at the literature on Technorati, and it seemed to be an open space. I assume I was probably the first person to write about this.” As many have observed, we speak differently in public and private spaces, employing different registers. In a private register, you might say, “I think that guy’s a dick”. In public, “I respectfully disagree that guy” and people map it backwards to say “I guess Danny thinks that guy’s a dick.”

The privacy embarrassments we’ve all been seing come from having private online speech lifted out of context and brought to the public. When sources ask journalists to go off the record, they’re not saying “You can’t report any of this” – they’re saying “I’m going to speak in a private register, and you must translate into public register before repeating this.”

As television becomes more all-pervasive, Danny speculates, people are learning how to drop into public register very fluently. Turn on a camera, and they speak like spokespeople. Of course, what makes reality TV work is broadcasting the private register, which is still transgressive and exciting.

Eight years ago, Danny tells us he predicted that the private register would gain a new foothold. Of course, a few months later, thefacebook.com came into being.

Systems like Facebook have the potential to bring private conversations into public, but there are technical and social protections. Friends understand that there should be rules for sharing. Oddly, though, we trust these very personal conversations to a corporate entity we have no personal relationship with.

What’s common to many of these tools is that they’re terrible at search. It’s very difficult to cut through the waves of data on Twitter and Facebook and search for something specific. Ask the people who engineer technology for these companies and they’ll feign embarrassment about the quality of search, but it’s a conscious decision: they want to preserve the context of conversations and make it harder to yank statements out of context. (He offers a strange aside on David Bowie’s teeth. If you see David Bowie perform, and you’re used to slick nightclub singers, all you can focus on are his teeth and how awful they are… or were, as he finally jad them fixed. Search is David Bowie’s terrible teeth when you look at Twitter or Facebook…)

Danny declares Twitter as the most successful at letting people speak in private voice in public space. It’s become quite common for celebrities to swear on Twitter. It creates the idea of intimacy with celebrities in the conversation, and people appear not to be shocked by the tone of the speech.

The rise of commentariat, the people who comment on published documents, forces another clash of registers. The quality of newspaper comments is often discussed in terms of the values of real names versus anonymity versus pseudonymity – Danny suggests that it’s a clash of registers. When people talk about the nastiness of online comments, part of what they’re commenting on is on hearing private register in public spaces. Fortunately, some newspapers are learning where to draw firm lines – fortunately, the New York Times doesn’t allow commentaries on obituaries, for example.

Moving into the realm of the secret: Danny observes that high school teachers realize they’re sufficiently public figures that they take most conversations offline, rather than risk recontextualization and register clash. Yes, there’s the danger of the private becoming very public, as in hacker attacks where Lulzboat took Stratfor’s corporate communication public. But in some ways, what’s even more uncomfortable is the information we thought was purely ours alone, like our online purchasing behavior, becoming public, as in situations like Facebook Beacon.

He closes with his predictions for 2012:

- more swearing
- panicked actions to defend the secret and unrevealed
- growing fluency in negotiating the public register
- more suicides

On this last point, he notes that South Korea – which often precedes the US in online behavior – closed down internet freedom in the wake of pop stars committing suicide over online disclosures of speech. If Rebecca Black committed suicide over what people say about her online, he speculates, we’d see a chill on online speech in the US.

After weeks of unseasonable temperatures, western MA finally got some snow yesterday morning, just enough to make me miss my train to New York City. So I was late for Microsoft’s Social Computing Symposium at ITP at NYU, missing my friend Dina Mehta’s talk. So I’ve been thwarted in my ambitions of blogging all the conversations taking place here, and I’ll instead offer some snippets of talks I caught.

Jenna Burrell studies cybercafes in west Africa, particularly in Ghana. So she was very interested when a wave of stories about “the dark side of the information age” reported on corrupt recyclers selling containers of used computers to unscrupulous dealers in Ghana and Nigeria, who dumped the machines into local waste facilities, causing serious environmental harm.

This didn’t read quite right to her, as she’s been studying “the career of the obsolete computer in Ghana”. The computers in most Ghanaian cybercafes are reused computers, Pentium 3 or 4 series. They frequently come with property tags – she shows us a CRT monitor with tags identifying it as the property of the US Environmental Protection Agency. While there’s probably a fascinating story about how that monitor made it from the EPA to an Accra cybercafe, she makes the point that it’s a working monitor – it’s been reused, not recycled. It’s not in a dump, it’s in active use.

There’s not a direct channel from the port to the dump site, she suggests. Second hand computers work their way through the economy. The best used computer dealers identify lots of machines with the same configuration and appearance so they can sell higher quality, tested goods to businesses and cybercafes. Other dealers work on the lower end, selling individual, unmatched computers. The machines that don’t work at all are sold to scrap metal dealers, mostly members of the Dagomba tribe, a northern tribe that tends to be economically disadvantaged in Accra.

Do computers end up in the dump? Yes. But it’s not as simple as the dumping of ewaste in Ghana, where waste is being inflicted on poor people. It’s people’s desire for computers, a legitimate desire, that creates a complex commercial ecosystem.

Samantha Doerr helps us understand what the Microsoft Digital Crimes Unit does. The answer: they take down botnets, and they spend a lot of time fighting child sexual exploitation.

In the time before the internet, she tells us, child porn was not very common. You might be a creep, but it’s very hard to find other creeps to share pictures with. While she’s careful not to condemn the Internet, Doerr notes that child porn is getting much more common, as well as more extreme and violent. A man was recently arrested in Seattle for posession of more than a million sexual images of children. It’s becoming more common to find images of infants of toddlers… because they can’t tell anyone about the abuse they’re experiencing.

Doerr’s strategy is to make it more difficult to share child porn. Her chief weapon is Microsoft’s Photo DNA technology. Photo DNA creates a hash of an image that can match other images even when the format changes or the image is being resized. Her team has identified some of the worst child porn images, ones where the children exploited have been identified, are confirmed as being under 13 and are being abused. Microsoft now checks these hash signatures against photos uploaded to Skydrive, indexed on Bing or transmitted by Hotmail, and Facebook is announcing adoption of the same possibility/

Doerr wonders whether we can win against child pornography. Microsoft recognizes the complexity of the challenge, and has just issued an RFP for research on the topic. Her goal is to change the dynamics of the equation. Child trafficking is on the rise because it’s currently more economical than selling drugs – if we can make child exploitation more difficult and less profitable, that would be a win.

Del Harvey has built the safety team at Twitter, working since 2008 to eliminate spam and other forms of abuse from the service, while trying to respect user needs. Working on the front lines of the service, she has a unique perspective on “unintended usage and unexpected consequences”, or as she puts it, “users do the darndest things”.

Many of the behaviors we associate with twitter – retweeting, hashtags, @reply messages – were not created by Twitter’s programmers, but were emergent behaviors created by Twitter users. When users start doing something novel on Twitter, it’s her job to look closely at the new behavior and ask, “Should you be doing that? How are you doing that?”

Where this job gets truly tricky is when users engage in behavior likely to get them suspended by Twitter’s automated algorithms. If you message someone multiple times, are you engaging with the, or harassing them? It might be one thing if someone messages you a dozen times, and another if they message a celebrity a dozen times – a form of showing their devotion and fandom. Some people send themselves multiple @replies, using Twitter as a form of bookmarking.

The easiest way to eliminate spam is to identify spammy URLs and block people who retweet them. But this works very badly when people retweet spam and add snarky comments to it. “Nothing pisses off a user as much as complaining about spam and suspending them for spamming”

Why do some users take all the trending topics and put them into sentences? Del isn’t sure, but it’s become a pretty popular practice, and it makes it unwise to block people who simply use lots of TTs in a post. Sometimes her team is able to anticipate behaviors – it seemed likely that people would try to report users as spammers to silence them. (Twitter has systems in place that makes this unlikely to be effective.) But what do you do with users telling Twitter to report their accounts as spammers, a behavior that’s unexpected and inexplicable.

Del’s talk gets a lot of laughs of the “users do the darnedest things” variety, but there’s a serious message. Her job, as she thinks of it, is to “try to figure out when users are experiencing unintended negative consequences” and mediate the consequences.

In an Ignite talk, Alex Leavitt offers a great example of the ways in which media is moving from individual platforms to existing in ecosystems. He introduces us to Hatsume Miku, an open source fandom and culture based around a vocal synthesizer program. The character of Hatsume Miku is a teal-haired anime popstar, whose songs are written by an army of fans who record her music, build complex music videos for her, and throw concerts in the physical world featuring the best of those videos. It’s hard to understand the sheer scale of the phenomenon – Leavitt notes that Hatsume Miku just appeared in Japanese Playboy, both in drawn form and as the photographs of a leading live action Hatsume Miku cosplayer.

The video system, built around a program called Miku Miku Dance, is one of the most stunning aspects of the phenomenon – Leavitt tells us it’s the #1 3D software package in Japan. Point a camera at you and your friends and you’re converted into Anime characters which move their mouths and limbs in sync with your actions.

The ecosystem exists through an integrated commenting and attribution system that allows people to publish on appropriate platforms, like YouTube, while ensuring followers of the community know about the individual publications.

Always the provocateur, Clay Shirky is predicting the demise of another industry: street level retail. His argument begins by noting the similarity of streetscapes in New York City, a repeating loop of drug stores, mobile phone shops and banks. As higher end businesses move to selling primarily on the web, lower-margin businesses move into retail space, a process that can’t continue forever.

Shirky suggests that New York made two major errors in repurposing urban space. The first was in insisting that loft space, used to manufacture products like belt buckles, must continue to be zoned industrial, just in case the belt buckle industry returned to the city. It took thirty years, he notes, before New York loosened those restrictions and let first artists, then ordinary people live in loft space. The second transformation has been the disappearance of the working waterfront. For years, New York was a center of global shipping. But in the container age, that shipping has moved far south of the city, and New York took a long time to realize that infrastructure dedicated to shipping needs to be repurposed into waterfront open and green space.

If street level retail is dying (and here, I assume, Clay will write something at length making a compelling case for this, as his 5 minute version is pretty hasty), will we react quickly enough to fill the spaces? Clay remembers purchasing comics at his local comic shop. It wasn’t a great retail experience – the selection was small – but it was a great community experience, an opportunity to gather with other similarly oriented nerds. Can cities like New York figure out how to transform street level retail into street level community space?

Mimi Ito wants help solving the problems of education. She notes that there’s a 50% high school dropout rate for black and latino youth, and reminds us that this isn’t okay – it’s creating problems of social stratification and inequality that we’ll be facing for years.

The sort of folks in this meeting are the educational 1%. We are learning elites who know how to mobilize the internet and develop professional identities. To help students engage with education, we need to help them develop the same sort of skills we rely on.

Ito has been interviewing people who learn by exploring passions online. She tells us of a webcomics creator, who while he attended college, taught himself what he knew about creating comics from his online encounters. He discovered the medium online, developing a passion, and began learning to create by following tutorials and how-tos online. In the process, he connected with a community of the likeminded and passionate. Ito calls this “connected learning”, learning in which embracing your passions allows you to connect with others and learn with them.

The Internet has lowered barriers to acquiring knowledge and expertise, but kids often have not deciphered the puzzle. We need to build better platforms that connect people around interests. Ito suggests that while Facebook connects you with the people you went to school with and Twitter with the folks you wish you went to school with, we need infrastructure that connects you with the people you want to learn from or want to teach.

Andrés Monroy Hernandéz studies the use of social media in conflict situation. He’s especially focused on narcoviolence in his native Mexico, and notes that in the country, he’s seen increasing adoption of social media aligned with an increase in stressful situations. In cities like Monterrey, not only is drug violence an everday occurance that impacts bystanders, it’s a force so powerful, it’s driven traditional media away. Reporters will not cover drug violence for fear of being killed or kidnapped. As a result, people are using Twitter and Facebook to create immediate alerts of violence in specific cities and neighborhoods.

This means that when you leave your house for work in Monterrey, you check a twitter tag like #mtyfollow to ensure that there’s not an active “balacera” – shooting – on the path you plan to take. Hernandéz has collected 300,000 #mtyfollow tweets and shows us a quick overview – the language is a language of violence and warnings. It’s centered on a very few people who consistently tweet about breaking news and others who amplify the stories.

Those using social media to report narcoviolence in Monterrey face at least two enemies. The government is worried about control over information and recently jailed two Twitter users for allegedly spreading misinformation. The cartels themselves are killing people who are using social media to document their actions – he shows us a banner hung next to the head of a Twitter reporter, warning others not to use social media to track drug violence. Citizen responses are not totally impotent in the face of these attacks – a group called CIC is using Ushahidi to collect and track tweets, offering a graphical map of violence in the city and a portrait of life during wartime.

Vastly more good stuff that I was able to cover in one post. Looking forward to today’s talks (right after the one I give this morning…!)

Robert Neuwirth is bringing new insights to familiar (for him, unfamiliar for most of us) territory in his book, “Stealth of Nations“. His previous work, “Shadow Cities” was a plea to take squatter cities and informal settlements seriously, rather than dismissing them as slums. (My review of Shadow Cities is here.) His mission in this new book is for us to reconsider the “informal economy”, which he rebrands “System D”.

“System D” is an abbreviation for “l’economie de la débrouillardise”, a tern coined in French-speaking Africa to refer to a system of “resourceful and ingenious” people who make their livings outside the formal, taxed and regulated economy. Neuwirth rejects the term “informal” because the coiner of that phrase, British anthropologist Keith Hart, included the criminal underground in his term, “the informal economy”. Neuwirth wants to celebrate the energy and ingenuity of people who make their living outside formal economic structure, but distinguish those he celebrates from those who are selling drugs or running prostitution rings. The heroes of System D may avoid taxes, smuggle goods or operate without permits, but Neuwirth sees them not as criminals but as hardworking people trying to make a living in systems that are broken and corrupt.

Neuwirth’s great strength is as a traveler and storyteller. Like “Shadow Cities”, “Stealth of Nation” is packed chock full with stories from the communities he’s visited in Brazil, Paraguay, Nigeria, China and the United States. We meet street merchants selling pens and cakes in São Paolo, a handbag manufacturer in Guangzhou and the baker of high-end (if unlicensed) olive oil cake in New York City. He takes a particularly deep dive in Lagos, a megalopolis he describes as “a System D city”, where virtually no infrastructures are provided by the state, and where basic services like power, drinking water and public transit are provided by private industry and workers’ collectives, who build systems that function with limited licensing, taxation or oversight.

This wealth of narratives helps make the case that System D is massive and pervasive. Working from numbers from the World Bank and using the insights of Austrian economist Friedrich Schneider, Neuwirth offers an estimate that System D is responsible for roughly $10 trillion in goods and services bought and sold annually. That makes “Bazaristan” the second largest economy in the world, behind the United States. He further argues that System D provides employment for a majority of adults in many developing nations. Whether or not we approve of the activities of System D, Neuwirth argues, we need to take it seriously because of the large number of individuals it impacts.

Neuwirth’s inquiry is extremely broad in scope, both in terms of the subjects he considers and the timescale he examines. Chapters look at phenomena like piracy and counterfeit goods, and smuggling across international borders, which Neuwirth examines primarily via Paraguay’s Ciudad del Este, a urban center that exists primarily so Brazilian citizens – and merchants – can avoid paying taxes. To provide a historical context for these sorts of trade, Neuwirth calls on classical economists, including Adam Smith, as well as histories from the 18th century to demonstrate the ongoing demonization and dismissal of System D merchants. For me, these excursions into the past are less enjoyable that the wealth of contemporary examples he provides, though they’re helpful in establishing that System D is a very old system as well as a new one.

The danger in both of Neuwirth’s books is that he loves his subject so much, he occasionally celebrates it uncritically. “Shadow Cities” occasionally read to me as a marketing brochure for Brazilian favelas, suggesting we abandon traditional urban planning and invite urban entrepreneurs to rewire the electrical grid to meet their needs. “Stealth of Nations” is more careful, and Neuwirth engages with the ways in which Lagos can be a nightmare for the people who live there, not just a creative laboratory for urban innovation. At the same time, he urges us to take seriously the miracle that Lagos works at all, a miracle that can be hard to see underneath the diesel smog, caught in an hours-long go-slow.

This appreciation for the complex systems that compose System D can push Neuwirth towards a sort of conservatism that’s familiar to readers of Jane Jacobs. Neuwirth’s Robert Moses is Lagos State governor Babatunde Fashola, who Neuwirth lambasts for clearing street merchants from busy intersections and setting up formalized markets in inconveniently located parts of the city. Neuwirth is right to point out that Fashola, and other urban planners, have a tendency to undervalue the contributions of street merchants, and tend to propose unworkable alternatives to current systems. But celebrating contemporary Lagos in the ways that Jacobs celebrated the Lower East Side seems to miss two critical points. First, to the extent that Lagos works right now, it just barely works – Neuwirth acknowledges as much when he points out that some of Lagos’s most impenetrable traffic jams are caused by the tendency of merchants to turn roadways into markets. Second, Lagos is growing at a ferocious pace, and Fashola seems to be taking seriously the challenge of allowing the city to continue functioning as a megalopolis, likely to soon be one of the world’s largest cities. One possible response to Neuwirth’s criticism is to point out that Fashola was just re-elected with 81% of the vote in a poll most observers saw as free and fair.

Neuwirth is a journalist and documentarian, not an economist or an urban planner, and it may be unreasonable to ask him to solve the thorny problem of bringing System D and the formal economy into closer partnership. Neuwirth examines Hernando de Soto’s work on formalizing System D through property rights. De Soto’s most helpful intervention is the observation that the poor have wealth – homes, businesses, assets – but few ways to access them. By creating a paper trail, establishing ownership over houses and other real estate, de Soto argues that the poor can access their wealth, borrowing against their homes and using the loans to start new businesses. Neuwirth looks at de Soto’s native Peru and concludes that formalization hasn’t done much to help System D. The problem is the banks, who are perfectly willing to accept deposits from System D entrepreneurs, but unwilling to lend to them. Neuwirth’s anger is rightly placed, and his solution – that communities and governments need to demand that banks serve the communities they are located in, not just their shareholders – is timely and correct, even if difficult to implement.

The solutions Neuwirth offers for strengthening and legitimating System D are, by his own admission, modest in scope. Merchants should work together to regulate their activities, settling disputes within mediation mechanisms. They should take responsibility for the physical spaces they inhabit and work to make them clean and safe. They should consider systems that review product safety and ensure the quality of goods sold. Neuwirth isn’t opposed to regulatory involvement in this space – he looks closely at the “pure water” industry in Nigeria, where entrepreneurs drill wells, pump water and purify it under government standards before selling it in single-use sachets to thirsty customers. The system could be a health nightmare if minimum health standards are not enforced. The Lagos government can’t provide clean drinking water to its citizens, so it has found a way to work with System D to ensure that people have water and the water doesn’t kill them – for System D advocates, there’s potential in that story and a model other governments might follow.

But the pure water story also reveals the apparent limits of System D. “Pure water” usually won’t kill you, but it’s an environmental nightmare, as millions of nylon bags clog the Lagos sewers. It’s a wonderful thing that Lagosians can drink safe water, but a system where thousands of school-age girls sell sachets of water because you can’t drink the water out of the pipes isn’t a system any sane planner would advocate for. System D can get Lagos’s citizens to work, but it’s never going to build affordable and environmentally sound public transportation. If merchants follow Neuwirth’s advice, they may collectively buy bigger diesel generators, but they’re unlikely to fix Nigeria’s laughably inadequate power grid.

The people Neuwirth celebrates are – rightly – frustrated by their governments. They avoid paying taxes both because those taxes can be arbitrary and unaffordable, and because they see very few government services in exchange bought with those revenues. But governments need revenues to build infrastructures. And, as economist Paul Collier argues, they need taxes – and need to put those taxes to use in productive ways – in order to have legitimacy. System D seems like a local maximum in an equation – when it works well, it’s amazing what entrepreneurial people can accomplish against impossible odds. But the solutions created are convoluted and incomplete, and it’s reasonable to worry that System D may prevent more formal systems from providing more complete solutions to societal problems.

I don’t actually disagree with Neuwirth on this point – I wrote an essay some years back about incremental infrastructure, an idea I’d had from studying African mobile phone markets, that suggested that systems like power grids and roadways might be built by the cooperation of entrepreneurs when governments failed. My proposal suffers from the same weaknesses I’m criticizing Neuwirth for: it’s hard to see how a collective of merchants builds a railroad, and sometimes a railroad is what’s really needed for economic development.

But that’s an awfully big problem to demand that Neuwirth tackle – if you want to understand precisely how complicated that problem is, try this thought piece from Collier, proposing a possible solution to railroad construction in sub-Saharan Africa. Neuwirth’s job isn’t to solve the problems of System D. What he does – compellingly, readably, engagingly, and frequently, brilliantly – is give the reader a picture of how the world’s economies actually work, and a convincing argument that we need to respect and understand these economic systems. It’s a good read and an important book.

When you pick up Neuwirth’s new book, also consider grabbing a copy of Gordon Mathews’s “Ghetto at the Center of the World”, a remarkable ethnography of a single building in Hong Kong, Chungking Mansions. Chungking Mansions is a nondescript and somewhat run-down tower block in one of the more crowded corners of Kowloon. Inside is a remarkable market, where Chinese, Pakistani and sub-Saharan African merchants interact with one another in a microcosm of global trade. Mathews refers to this economic phenomenon as “low-end globalization”, and his book unpacks the history, mechanics, personalities and motivations in a way that is absolutely fascinating.

Chungking Mansions exists because of a peculiarity of Hong Kong’s visa policies. Tourist visas to Hong Kong are easily obtained by citizens of many nations – residents of countries like Ghana, Nigeria and Kenya often have difficulty obtaining visas to Europe, the US or China, but are able to travel to Hong Kong for anywhere between 7 and 90 days, depending on the discretion of the immigration officer. As China became a major manufacturing power, Chungking Mansions became a critical interface between Chinese factories and developing world markets. The upper floors of the building feature low-cost guesthouses that cater primarily to traveling merchants, and restaurants that offer home cooking for the African and South Asian migrants who work out of the building.

On the ground floor, dozens of stalls feature Pakistani merchants selling Chinese-made mobile phones to African middlemen. Mathews documents the trade in intimate detail, explaining the ownership of the individual stalls (they are generally rented from Chinese owners who are rarely present in the building, but have a powerful owner’s association that governs the working on the market), the provenance of the phones sold (including the difference between original phones, 14-day phones – original phones returned to the vendor by dissatisfied customers, good fakes and bad fakes) and the economics of importing phones into sub-Saharan Africa. Mathews posits (without much data to back this claim) that up to half the mobile phones in Africa come through Chungking Market and enter African markets through the luggage of entrepreneurs.

I found Mathews’s account so compelling that Chungking Mansions was my first stop when visiting Hong Kong a few weeks ago. Based on his explanation of Chinese perceptions of the building (as a dangerous place filled with drug addicts and criminals), I expected a much shadier place than I actually found. Chungking Mansions is immediately familiar to anyone who’s bought electronics in the developing world – it’s cleaner and better organized than markets I’ve been to in Nairobi and New Delhi, but in some ways, functionally the same place. Walking through the stalls, I experienced a tesseract, a folding of space that let me move between Hong Kong, Pakistan and West Africa over the course of a few meters. I dropped into one of the few non-phone stores, a clothing store featuring street fashions, including a wide array of Yankee caps. I gave the merchant grief about not stocking Red Sox hats, quickly figured out that he was Ghanaian, greeted him in Twi, and was warmly embraced and invited upstairs for fufu and groundnut soup. It wasn’t at all hard to figure out why Mathews had fallen in love with the place – if you’re interested in how globalization is transforming economies, Chungking Mansions really is one of the centers of the world.

I had the chance to meet Mathews when we lectured together at the University of Hong Kong a few days later. He’s as wonderfully crazy as you’d imagine him to be, and told me that he’d written the book in a bar across the street from his research site. “The key is that the bar has roasted peanuts in the shell. I’d shell a peanut and think, then write a sentence, then sip my beer. That writing pace is just perfect as long as you remain under three beers.” Rarely have I learned so much from a single ethnographer – how to smuggle phones into Ghana in my luggage, the best strategies for overstaying my Hong Kong tourist visa, how to befriend Nepali heroin addicts, and how to pace my writing.

I’ve been pushing Mathews’s book on the ethnographers I know because it’s an amazing example of the power of the deep dive. It’s possible that no one on the planet understands Chungking Mansions as thoroughly as Mathews does based on his decade of research. But his insights are profoundly helpful not just for understanding this one wonderful and strange building, but for understanding globalization as it is actually practiced. Where Neuwirth takes a broad view, considering economies on four different continents, Mathews rarely leaves the confines of a single building and still manages to tell a story that’s global in scope and impact.

Somedays it seems that everyone has joined Twitter. And then a new account comes along and raises interesting questions about what the service is for and how it should be used.

Welcome to Twitter, Harakat Al-Shabaab Al Mujahideen Press Office, now tweeting at @HSMPress.

Al-Shabaab is the militant organization waging war against the Transitional Federal Government (TFG, for short) of Somalia. They’re an offshoot of a more moderate Islamist organization, the Union of Islamic Courts, which was ousted by the Ethiopian military, backed by US support and intelligence. In the wake of their ouster, extremists associated with the movement reformed as Al-Shabaab (“the youth”) and have been fighting over control of Mogadishu and southern Somalia. In designating Al-Shabaab a terrorist organization in 2008, the US state department asserted ties between Al Shabaab and Al Qaeda – a backgrounder from the Council on Foreign Relations questions the strength of those ties, but points out ideological similarities between the organizations.

Recently, Operation Linda Nchi (Swahili for “Protect the country”), which involves the cooperation of TFG, Kenyan, Ethiopian and French troops, has been attacking Al-Shabaab positions within southern Somalia. The mission is a response to the kidnapping of Kenyan civilians and aid workers from the Dabaab refugee camp in Kenya, allegedly by Al-Shabaab. The mission has raised concerns about possible human rights violations, and some Kenyans have raised questions about whether their country should be engaged in a possibly lengthy war with their neighbor. Others have pointed out that attacks against civilians in Nairobi are likely to be linked to Al-Shabaab, pointing to the importance of ousting the militants.

Operation Linda Nchi has had a presence on Twitter since October 27th, when Kenyan military spokesman Major Emmanuel Chirchir began his Twitter feed.

Good to be here to give you the right and official information

October 27, 2011 7:24 am via webReplyRetweetFavorite

@MajorEChirchir

Major E. Chirchir

The major has been an avid correspondent, answering questions and inquiries. He’s also used Twitter in some very unusual ways, attempting to alert civilians in Somalia that their towns will be under attack:

#OperationLindaNchi BAIDOA, BAADHEERE,BAYDHABO,DINSUR, AFGOOYE,BWALE, BARAWE,JILIB,KISMAYO and AFMADHOW will be under attack continously

November 1, 2011 11:38 am via webReplyRetweetFavorite

@MajorEChirchir

Major E. Chirchir

#OperationLindaNchi The Kenya Defence Forces urges anyone with relatives and friends in the 10 towns to advise them accordingly.

November 1, 2011 11:39 am via webReplyRetweetFavorite

@MajorEChirchir

Major E. Chirchir

Whether this “please tell a friend” strategy will actually save lives is unclear. It’s fascinating (to me, at least) to see the electronic version of leaflets warning of incoming bombs…

So perhaps it’s not surprising to see Al Shabaab responding with a Twitter presence of their own. Major Chirchir’s feed features regular updates on military actions:

#OperationLindaNchi 3rd Dec; Attack at Hayo, 11 TFG soldiers died, 54 injured 25 flown to Garissa and Nairobi, 40 Al Shabaab’s killed

December 6, 2011 1:52 pm via webReplyRetweetFavorite

@MajorEChirchir

Major E. Chirchir

And so does Al Shabaab’s:

6-DEC: Mujahidin forces attack #Amisom base in Dharkenley District, #Mogadishu. 3-hour battle resulted in some #Amisom casualties+base burnt

December 7, 2011 12:47 pm via webReplyRetweetFavorite

@HSMPress

HSM Press

But it’s hard to imagine seeing some of Al Shabaab’s tweets on an official military feed:

7 Uganda-trained TFG soldiers surrender themselves to Mujahidin in #Mogadishu. They are welcomed after proclaiming repentance from apostasy

December 7, 2011 12:56 pm via webReplyRetweetFavorite

@HSMPress

HSM Press

That Al-Shabaab is using social media isn’t entirely surprising. They’ve developed a reputation for issuing well-written English language press releases, and had maintained a website, chat rooms and other web presences. (Many have been removed – it’s not clear to me whether they were removed by the decision of ISPs hosting them, or whether those companies came under government pressure.)

And Al-Shabaab has shown a willingness to use multiple media platforms to spread their message. Omar Hammammi, aka Abu Mansoor Al-Amriki, an American muslim who joined Al Shabaab in 2006, has an extensive collection of videos on YouTube, and has recently been releasing rap recordings, including “Send Me A Cruise“, a plea for martyrdom by cruise missile.

Needless to say, not everyone is thrilled that an extremist organization is using Twitter to disseminate its news:

Will al-Qaeda be allowed to use #Twitter? No. Then why is @HSMPress – Somali terrorists – going unchecked? Freedom stops with violence.

December 7, 2011 2:33 pm via webReplyRetweetFavorite

@Ed_Husain

Ed Husain

And it’s not hard to imagine that Twitter may end up fielding some questions from the US government, which has declared Al-Shabaab a terrorist organization, and might conceivably argue that providing Al Shabaab with a platform to express themselves could constitute “material support” to a terror organization. (It helps that Twitter is free for Al Shabaab to use, and that Twitter doesn’t review new users of the service.) There’s always an argument to be made that the account in question is not, in fact, Al Shabaab… there are several Al Qaeda accounts that appear to be fakes (including a very funny one retired on 9/11/11). Heck, there’s even a parody Major Chirchir account.

I’ve made the argument that corporations who make possible the digital public sphere need to support people’s right to speak freely, even when that speech is unpopular. (That’s the subject of the talk I gave at the Chicago Humanities Festival a few weeks ago.) That means ensuring that Wael Abbas can post about Egyptian police brutality on YouTube… and I believe it means making it possible for Al-Shabaab to share their views via Twitter, even while I condemn those views.

Anyone know whether there are other extremist, terrorist or militant organizations using Twitter in this way? How has Twitter reacted thus far? And if – as I suspect – Twitter is making their platform open to all types of speech, including inflamatory speech, do you agree with my argument that this is ultimately a good thing?

This afternoon, MIT’s Political Science distinguished speakers series hosts Regina Dugan and Kaigham Gabriel, director and deputy director of DARPA, the US defense advanced research project agency, who are here to speak about advanced manufacturing in America. The title for their talk is “Just Make It”, a response Dugan offers to people who ask her to predict the future. “Visionaries aren’t oracles – they are builders.”

She shows a five minute video of nerd porn, a montage of dismissive predictions about technologies (like Lord Kelvin’s statement about the impossibility of heavier than air flight, followed by footage of the Wright Brothers, and then from Top Gun. The video ends with observations about the time to 50 million users for different technologies is rapidly shrinking, pointing to Facebook’s sprint to 100 million users, and offers images of protesters holding banners celebrating the internet. “Still think social media is a fad?” the video asks. The video ends with a challenge for the engineers in the room – “just make it”.

Dugan tells us that the decline in America’s ability to build things is a national challenge, if not a crisis. Americans consume an increasing percentage of goods made overseas, and are less likely to be employed making things. Perhaps this reflects on productivity increases, or on currency manipulations, but it has implications, she warns, for national defense. Adam Smith warned that if an industry was critical to defense, it is not always prudent to rely on neighbors for supply.

There have been many years of debate around the inefficiency of America’s design and building of defense systems, Dugan tells us. One extrapolation of increase in airline design cost – sometimes referred to as “Augustine’s Laws” – suggests that by 2054, a single military aircraft will cost as much as the entire military budget at that time. Obviously, it’s dangerous to extrapolate linearly from current data… but if you do, the cost of military systems is growing much more rapidly than defense budgets. “Quite obviously, this is not sustainable”.

When we design aircraft, she tells us, we’re often designing ten years out. That means we’re trying to understand the threat environment ten years out. That’s risky. “Lack of adaptability is a vulnerability.”

What’s worse is that it’s really expensive. She shows a graph of production costs for the F-22 fighter. The price per unit keeps increasing, and the volume required keeps dropping. This might be because we need to amortize design costs over fewer units. Or it might be because the costs get so high, we simply can’t afford as many units as we wanted. This isn’t just true of the F-22 – it’s true for the Marine EFV project and the Comanche helicopter as well.

This difficulty in building complex systems has implications for defense and for the economy as a whole, she tells us.
“To innovate, we must make. To protect, we must produce.” DARPA is not a policy organization, she tells us, but pushing from “a buy to make strategy” is of strategic importance to the US Department of Defense.

There’s $200 million a year being invested in innovation, looking for ways to change the calculus of cost increase. Can we turn a long problem like vaccine design into one we can solve in weeks? Could we permit the participation of tens of thousands of designers into a process and harness their ideas? She suggests that the future of innovation is around increased speed of production and number and diversity of designs. The rise of electronic design aides revolutionized the semiconductor industry – could this shift in speed and diversity bring a similar paradigm shift?

Dugan tells us that the systems we have to manage complexity are inherited from 1969-era systems engineering. We take complex systems and split them along functional lines – power system, control system, thermal system – then try to put them back together. What happens is that we experience emergent behaviors that weren’t predictable. As a result, we end up with a design, build and test system that we iterate through, trying to solve those emergent problems.

This isn’t the only way to design complex systems. She shows a graph that measures time to design, integrate and test, versus a measure of product complexity, which includes part count and lines of source code. There’s a linear increase in time to build to complexity for aerospace defense systems. Another piece of the graph shows a flat design and test time cycle with increasing complexity – that’s the semiconductor industry. And a third industry – the best in class automotive manufacturers – show a decrease in time with an increase in complexity! How are they pulling this off?

Gabriel tags in here, to explain how the semiconductor industry achieved gains in complexity without extending the timeframe necessary to design and test their products. The key factor was a decision to control for time. “If we aren’t out there with new chips in 18-24 months, we’ll miss the next generation of PCs.” So the principles of VLSI design were optimized around producing new product on a timecycle as tight as that for less complex integrated circuits.

Two major design innovations characterize the VLSI shift, Gabriel tells us. First, it’s critical to decouple design and fabrication, a shift that was comparatively easy for circuit designers to accept. The second was initially heresy: you needed to stop optimizing each transistor, and sacrifice component performance for ease of system design and reliability.

We’ve seen a similar move in computer programming, a shift away from assembler, which produces very efficient code that’s hard to test, to higher level programming languages. Those languages abstract operations, which leads to a decrease in performance efficiency, but since we’re no longer as limited by how many operations a computer can perform, the design speed benefits outweigh the performance compromises. He hints that we may be seeing some similar shifts in biological sciences as well.

How does this work in terms of DARPA projects? Dugan retrieves the mic to speak about the Adaptive Vehicle Make program, designed to build a new infantry vehicle in two years instead of ten. A first step is developing a language to describe and design mechano-electric systems so they can integrate more smoothly. The vehicle, she tell us, will be flexibly manufactured through a “bitstream-configurable foundry-like manufacturing capability for defense systems” capable of “mass production in quantities of one”.

With facilities that can accept a design and custom-forge parts, she believes we can move to an increasingly democratized design system, which enables the participation of many more people to design and submit systems to foundry-like fabrication facilities. We’ll design vehicles “using the most modern techniques of crowd infrastructure and open source development,” in a program called VehicleForge.mil. (While a valid URL, there’s no webserver at that address. Just wanted to save you a Google search or two.)

Critics tell her this approach won’t work. But Dassault recently designed the Falcon 7x aircraft using “digital master models, by tail number, for aircraft” – i.e., building extremely complex individual models for each aircraft they build. The models only do geometric interference (i.e., they test whether the parts fit together), but they’ve halved the time needed to produce a new plane. Critics claim that the analogy between integrated circuits and military vehicles is an inept one. But in terms of part count, ICs are much more complex than vehicles. What’s complex is the diversity of components used in the combat vehicle.

A new experiment, conducted in cooperation with Local Motors, a small-scale vehicle fabrication company (see my notes on the founder’s Pop!Tech talk in 2009) invites designers to compete to design a combat support vehicle, the XC2V. $10,000 in prizes were offered, and instead of getting the 3 designs they get in an invitation-only design scenario, they received 159, 100 of which the judges deemed “high calibre”. It wasn’t a clean sheet of paper design – the chassis and drivetrain were designed by Local Motors – but it was effective at expanding the idea pool, and led to a functioning design within four weeks.

The power of the crowd may be even greater in a field like protein folding, where humans are still able to solve some problems better than algorithms. Foldit is the brainchild of a biochemist, a computer scientist and a gamer, who decided to turn protein folding into a game, building “a Tetris-like environment for folding”. 240,000 people have signed up to play, but what’s really cool is “the emergence of 5 sigma savants for protein folding, some of whom have very little biochemistry training.” Recently, Foldit solved a key protein – a retroviral protease SIV for the rhesus monkeys – which had been unsolved for 15 years. The community folded it in 10 days. Projects like this, she tells us, make her a believer that bringing many diverse minds to a problem and increasing the pace of building will increase the speed and diversity of innovation.

Gabriel offers three other examples where massive innovations are possible through new methods.

Optics are the dominant cost in many imaging and sensor systems. It turns out that making light do something different – bending, focusing, diffusing – requires materials and systems that are heavy, complex and expensive. M-GRIN – manufacturable gradient index optics – moves beyond lenses that are made out of a single material with a single index of refraction. Instead, they use a stack of multiple layers and films, combined via heat and pressure, to make lenses that are smaller and lighter. A test around a shortwave infrared lens produced a device that was 3.5x smaller and 7.5x lighter. That’s a breakthrough… but the real innovation is creation of a set of design rules that let you go from an application to a recipe for combining materials into the lens you need.

In telling us about maskless nanolithography, Gabriel tells us “Moore’s law is dead in circuit design, though the corpse doesn’t know it yet.” The culprit is heat – we can make tighter and smaller circuits, but they’re getting very difficult to cool. As critical is cost. Working at ultra-small line width is prohibitively expensive. It’s hard to spend tens of millions on a set of 45 nanometer masks to create a few hundred chips for a defense system, when building those masks costs tens of millions of dollars.

We know how to do lithography without masks, but it’s traditionally been very slow. So now designers have built a system that creates and bends an electron beam, then splits it into millions of beamlets, controlled by a “dynamic pattern generator”. Program that pattern generator, and it allows millions of writing operations to happen at the same time, leading to a current working speed of 10-15 wafers per hour, the minimum required to produce custom ICs for military applications.

His third example is the accelerated manufacturing of pharmaceuticals, a strategy he tells us was Plan B in 2009-2010 if the H1N1 flu virus had resurfaced. It’s very hard to produce vaccines quickly – egg-based strategies require a piece of virus and many thousands of chicken eggs. These methods work, but can require 6-9 months to build up a stockpile. A new method uses tobacco plants to produce custom proteins, working from strands of DNA in the virus. Envision a football-field sized building filled with lights and trays of tobacco plants. A facility like that can now produce a million doses a month of a novel vaccine. In scaling up capacity to 100 million doses per month, the key problem turned out to be lighting – it was impossible to light everything without switching to LED bulbs. Once they made the switch, they had a new opportunity – tuning the spectrum to optimize production. Using an experiment of “high school science complexity”, they grew plants under different lighting conditions for a few weeks, and determined a mix of blue and red frequencies that doubles protein production.

Gabriel ends with a slide quoting MIT scientist Tom Knight:

“The 19th century was about energy.
The 20th century was about information.
The 21st century is about matter.”

If we embrace this challenge, Gabriel tells us, we will be able to make things at the cost we used to produce and stockpile them in bulk, and this change will change how we innovate.

I thought the DARPA folks gave an impressive talk, inasmuch as they got me thinking about a problem I’d not considered – the insane cost and time frame of producing military equipment. But for a talk sponsored by the political science department, it seemed woefully lacking of discussions of politics or markets. If I were trying to explain the difference in production processes between military vehicles, consumer automobiles and integrated circuits, I suspect I might look at the power of markets. IC manufacturers needed to build chips quickly because customers wanted to buy newer, faster chips… and would buy other chips if the manufacturer wasn’t fast enough. Ditto for automobile companies.

The defense industry is different. It’s very hard to terminate a weapons system, even if it’s massively over time and over budget. The competition happens well before a product is built. Discovering that the F-22 production isn’t going well doesn’t create a market opportunity for another company to produce a better product faster – the company producing the F-22 is going to get paid, even if they take an absurd time to produce the product.

I admire the approach Dugan and Gabriel are putting forward, and certainly appreciate that it plays well to a room full of engineers. But I was very surprised not to hear questions (and I only caught the first five or six) about whether the DoD purchasing process can be reformed so long as military budgets are sacrosanct. We’re currently facing mandatory budget cuts with the failure of the budget supercommmittee, and conventional political wisdom suggests that the social service cuts will go through, while the defense ones will not. How do you encourage companies to innovate when they’re currently amply rewarded for dragging design and production out over decades? How do you innovate without market pressures?

My homogeneously left-wing family was talking politics over the Thanksgiving dinner table and realized the solution to America’s current social problems was to simply adopt the Egyptian political system – let the military run everything. The right doesn’t like cutting military budgets, but is okay when the military provides state-sponsored healthcare and subsidizes education. All we need to do is ensure all Americans are employed by the US military and we can build a thriving, successful welfare state. The same absurdity behind that suggestion is what makes DARPA’s ideas so hard to implement – if there’s no pressure to cut military budgets, anything is possible… except real innovation around cost and efficiency.

There’s a great blogpost from Nancy Scola about the rise of Twitter hashtags as form of political discourse, specifically focusing on the #WeCantWait tag, which both quotes President Obama about the need for rapid action on a jobs bill, and invites snarky commentary on both sides of the political aisle about what Americans can’t wait for (a one term Obama presidency, a more cooperative Congress, etc.) Scola steps right up to the line of coining a neologism – the snarktag – with this observation: “Once the Dewey Decimal system of Twitter, hashtags are being embraced by the political class as an ideal way to snark.”

I mention the piece for three reasons. It’s a good read. It quotes Gilad Lotan of media analysis firm Social Flow at length, and Gilad spoke this evening at MIT, along with Hal Roberts and Erhardt Graeff on “Mapping Media Ecosystems“, an event I hosted for the Center for Civic Media, which I’ll blog about tomorrow. (Video will be up shortly – very cool event.) Third, Nancy’s piece got me thinking about another related, unnamed Twitter phenomenon that I’ve been experiencing: the mass rebuttal tweet.

Since the start of the Bahrain uprising in February of this year, I’ve been tweeting about Bahrain fairly often. I tweeted about the disappearance of Global Voices blogger Ali Abdulemam, and his sentence in absentia to 15 years in prison for his alleged role in plotting a coup against the government. I’ve tweeted about my frustration that the US continues to station a large contingent of military personnel in Bahrain and was close to selling armored Humvees and missiles to the country. (Under political pressure, the Obama administration has delayed that sale.)

When I tweet about Bahrain, I get fairly few retweets – it’s not an issue many people are following. But I started getting regular responses from @fatoooma92. This user identifies herself as a “Student @ CHS year 2″, which likely refers to the Bahrain College of Health Sciences. Much of her stream is in Arabic, but responses to me are in English, and they argue in passionate, if unpersuasive, terms that Bahraini protesters aren’t peaceful activists, but dangerous, violent traitors.

Fatoooma92 is fond of sending videos and images to make her case. While I don’t find them especially persuasive, evidently she does. And she sends these videos to a wide range of people who’ve written about Bahrain: not just me, but Barack Obama and Nick Kristof.

This is a little different from a now well-established Twitter practice: hijacking hashtags. If I want American conservatives to know about a story I think they’ll like (or hate), I can tag it #tcot (Top Conservatives on Twitter) and people following that tag will stumble on my link. (Yes, posting bit.ly links to The Lemon Party to #tcot has been tried, and no, it’s not all that funny. Besides, do it enough and conservatives will post their own disturbing links to #p2 – progressives 2.0 – or worse, to the universal liberal hashtag, #npr…) These rebuttal links aren’t going to the #Bahrain conversation, which has at least two sides to it. It’s a personal message, visible to only the targeted individual (and someone who happens to be following both the sender and the recipient.)

As Fatoooma92 is sending the same message to lots of people, it looks a little like spam. But it’s not commercial. And to a certain extent, it’s not unsolicited: I’ve posted using the tag #Bahrain, and Fatoooma92 is engaging with me directly, as someone who’s expressed an opinion on Bahrain. Unlike broadcast media in America, which abandoned right of response in the scrapping of the fairness doctrine for most new stories, Twitter ensures a right of response. Don’t like something I say? You can send me an @message, and there’s a decent chance I’ll read your response.

On balance, I think this is probably a good thing. Yes, it’s possible that Fatoooma92 is not a real student in Bahrain, but the astroturf creation of a PR agency attempting to defend Bahrain’s reputation in Twitterspace. (If Bahrain doesn’t have a firm attempting to contest perceptions in social media, it’s probably just a matter of time before they find one.) And this sort of activity reminds me more than a little of Zumabot, an early bot that trawled Usenet for references to Turkey and automatically posted rants accusing Armenians of genocide against Turks in WWI. (Zumabot had the odd habit of not being able to distinguish between the country and the bird, so discussions of Thanksgiving cookery had a tendency to become filled with anti-Armenian hate speech.)

But it’s also possible that Fatoooma92 is a real person, who really thinks I don’t understand Bahrain and am being brainwashed by a global media conspiracy. Whether or not she’s right or wrong is, to some extent, irrelevant. In the same way that it’s helpful for me to get pushback (as well as reinforcement) when I amplify a story like Morgan Housel’s argument that Occupy Wall Street protesters are likely to be part of the globally economically privileged 1%, it’s important to get the reminder that what I believe about Bahrain is not universally believed, and that other people are at least as passionate about the topic than I, often in a different direction.

The flip side, of course, is that being on the receiving end of this speech is pretty unpleasant. I checked in on my @messages while writing this post, and was greeted with this missive from @perrysupport129: “@UBCSMN @EthanZ Anti-Christ to Muslims: You’re filthy cowards and Muhammed was a child molester” followed by a URL, making the “argument” at more length. The trigger for this rebuttal appears to be the fact that I’m giving a talk on the Arab Spring at the University of British Columbia (@UBCSCM). Perhaps a Rick Perry supporter is searching for every mention of the word “arab” and tweeting offensive screed as a response. Or perhaps someone wants to portray Perry supporters as ignorant racists, and is creating accounts like this one to make the case. Again, it’s hard to know.

There’s some sort of psychological impact that comes from receiving a rebuttal tweet. Twitter is a social network, and to some extent, we’re all looking for the small serotonin burst that comes from an affirmative retweet – “Yay, a person liked what I have to say!” Not only does the rebuttal fail to provide the boost – it provides (for me, at least) a much stronger negative signal: someone I don’t know disagrees with me strongly enough to single me out and correct me. Did I get my facts wrong? Is this a chance to start a discussion, or is someone merely yelling at me? Even if I’m confident about what I wrote, the rebuttal tweet interrupts my comfortable echo chamber of affirmation and invites me to think about whether I’m considering an issue broadly enough. And that’s often a good thing.

Except when it’s not. I have friends who are knowledgeable about Israeli/Palestinian relations who choose not to write about the subject because they fear a flood of tweets, messages and blogposts in rebuttal. Many of those responses aren’t meant to convince – they’re meant to bully the initial speaker into silence. And perhaps that’s what Fatoooma92 is trying to do. Her first tweets made clear that she, as a Bahraini, knew more about what was happening in her country than I did, and that I should butt out. Had I not been following Bahrain closely, I might have taken her hint and shut up. It seems to me the value of the practice is directly connected to whether it’s attempting to silence speech, or attempting to challenge opinions expressed. Which direction it evolves in, and whether the practice remains fairly obscure or becomes commonplace: I look forward to watching and finding out.

Designer Richard Vijgen has posted a lovely and ambitious visualization of a data set that has special personal meaning for me: 650 gigabytes which represent tens of millions of homepages hosted by GeoCities before it was shut down by Yahoo! in 2009. From 1994-1999, I worked at Tripod.com, leading competitor to GeoCities. When our larger, more successful competitor shut down in 2009, some of the people who’d been involved with founding Tripod traded emails, congratulating ourselves on the fact that our site still exists and still hosts homepages, even though ownership of the company has changed hands several times. But that celebration was hollow – we may built one of the ancestors of today’s participatory media platforms, but our glory days are long past.

The Deleted City from deletedcity on Vimeo.

Vijgen describes his project – which visualizes the filesystem of the GeoCities archive as a vast city – as a form of “digital archeology”. It’s an interesting term to use – while archeology is the study of civilizations through the study of their artifacts, it’s often associated with the study of long-dead civilizations. GeoCities is an abandoned city as much as a dead one. Yahoo! shut it down after concluding that there wasn’t enough traffic to the millions of homepages to justify selling ad inventory on them or continuing to pay the server upkeep and maintenance costs. (My guess, based on helping run a similar company – removing copyrighted content and dealing with abuse complaints was likely another major cost for Yahoo!)

People stopped caring for their pages and moved on. The pages persisted, as digital things do, unchanged from the last time they were tended to. (At a conference in Chiba in 2005, a scholar told me that a term had emerged in Japanese to refer to abandoned blogs. The term was similar to the word for “river stone”, implying something solid, unmoving and mute.) As an archeology project, visualizing Geocities is more a study of Centralia than Chaco Canyon: we know who left and why. If it’s worth studying the structures left behind, it’s not to solve a mystery. It’s to understand the shift that’s taken place.

“Ten years later in 2009, as other metaphors of the internet (such as the social network) had taken over and the netizens had moved on to Myspace and Facebook, Geocities was shut down and deleted.” The shift in paradigm that Vijgen describes has two dimensions. He’s talking about a shift from a geographic metaphor that Geocities was nominally organized around, where people with similar interests located in the same “neighborhoods”, lovers of rural life in “Heartland”, gays and lesbians in “South Beach”, etc. (We studied GeoCities neighborhoods at length at Tripod and decided that affiliation was little more than nominal – there was no zoning that prevented off-topic pages from appearing in the “wrong” neighborhoods, and very little ability to predict what your “neighbors” were interested in.)

The more important shift in metaphor was from pages to streams. In the mid-1990s, we understood the web in terms of pages. Some pages were meant to be permanent, others changing, others completely ephemeral. Blogs updated the paradigm somewhat – they were pages we expected would change, daily, perhaps weekly. But they were pages, with permanency and permalinks. And you controlled what went on them, even if you permitted comments on your blog. Conversations took place between spaces – I link to you, you link to me. In the age of Twitter and Facebook, pages feel too permanent, too fixed. You produce a stream of updates which flow past your friends. If they follow you closely, they might hang on your every update – more likely, they dip their feet into the stream now and again, seeing what you’re up to, chiming in with a comment or an upvote.

It would be a mistake to visualize these interactions as buildings composing a city. They’re rivers, distinguishable by path and magnitude, but shifting and ever-changing. You can never step into the same lifestream twice.

We weren’t total idiots in the mid-90s. We knew that pages weren’t the right metaphor, weren’t going to be state of the art forever. We could see communities emerging in systems like Webrings, which tied disparate pages together into a loose aggregation. (When Charley Lanusse, the creator of Webrings, wisely rejected our overtures to buy his company, Tripod built “pods”, our version of the same basic tech. Charley later sold his company to GeoCities…) But knowing you’re a purveyor of a dying paradigm isn’t the same thing as knowing what the next big thing will be, or being able to build it.

Paradigms can shift quickly on the Internet. It’s hard to imagine anyone unseating Facebook, especially given the limited traction Google+ has achieved despite valiant efforts. But not everyone thought Yahoo! was insane when they paid billions for GeoCities, or when Murdoch bought MySpace. That sense that Facebook isn’t quite what we want, blurs boundaries between communities we want to keep separate, isn’t respectful enough of our privacy or our ownership? To me, that’s an indication that the paradigm is not quite right and ready to shift. To what?

I have no idea. But that sense that the ground is moving would make me reluctant to invest too much time, money or energy in Facebook. (Then again, it’s worth remembering that Tripod made lots less money than GeoCities, in part because we got out too soon. We weren’t making money selling ads on homepages, couldn’t see that changing and figured we’d better get out when we could. GeoCities waited longer and sold for much more. So did much smaller, less successful companies like TheGlobe.com. Knowing the ground is shifting isn’t necessarily good for your fiscal health.)

I chaired a panel at Microsoft Research the other day on privacy, where three very smart researchers offered their takes on what we do and don’t understand about online privacy. (Chris Conley of the Northern California ACLU summed it up nicely: “We say we care about privacy. But that’s not how we behave.”) In the conversation over drinks that followed, we got the inevitable question about big data: is there any escape from the masses of data that marketers are collecting about our every move?

I’m starting to feel like a contrarian on this question. Yes, my browser is riddled with cookies, and yes, ads for treadmills now track me across the web because I was browsing exercise equipment on Amazon the other day. But I’ve been less worried about this since Doc Searls pointed me to Rapleaf, a company that claims it “wants every person to be able to have a meaningful, personalized online experience.” It achieves this meaningful experience by selling your personal data to advertisers so they can better target ads to you. And they’ll let you retrieve and review the info they have about you. Despite the reams of personal information I’ve posted on this blog and social media sites across the web, Rapleaf believes I’m an unmarried, childless, underemployed likely Republican voter.

Should I worry about companies storing my data if they seem incapable of drawing correct inferences about me? Is the agglomeration of personal data more worrisome than the very real tendency of old bits to decay and disappear? GeoCities survives only because the Archive Team sprang into action to save it. Other sites and communities expire every day. Are we being wise, or paranoid, when we assume that our movements will be tracked forever in massive databases, while our utterances and creations have a tendency to expire and disappear?

Seeing GeoCities as an archive and an art project makes me feel old. In the dot.com days, people talked about internet years as if they were dog years – a company around for 4 years was a stable mature adult of 28, and an eighteen-month old startup might be worth taking seriously as if it were entering its early teens. This blogger makes an interesting case for 4.7 internet years to a calendar year – by his math, I’m over 100, fully entitled to shoo these twittering youngsters off my lawn with my cane and to wander GeoCities long-dead neighborhoods, remembering when this used to be a cool and trendy neighborhood.

Does history slow once the Internet is no longer the province of alpha geeks who decamp for greener pastures as soon as they’re mapped? Once we’re all on Facebook, will we never leave?

Take it from an old man who’s watched tumbleweed roll through the streets of a deleted city. Everything changes. Nothing lasts. That’s a good thing.

“Dreams seen by a man-made machine
How does it seem, how does it seem
That we can see each others dreams”
- CAN, “Last Night’s Sleep” from Until the End of the World soundtrack

It’s a rainy Friday here in western MA, the perfect day to curl up with some scientific papers. There’s two papers sparking a lot of online discussion today. One is CERN’s fascinating announcement of neutrinos that appear to travel faster than the speed of light. I know perfectly well that I’m not going to understand that paper, so I’m waiting for Chad Orzel at Uncertain Principles to explain it to me. (After all, he’s the guy who explains relativity and quantum mechanics to his dog – I suspect he’ll get me to understand the paper and the controversy.)

Another is this fascinating paper from the Gallant Lab at UC Berkeley, where the authors experimented on themselves, imaging their brains using functional MRI processes while they watched movies of the natural world. By analyzing the blood flow to their brains as they concentrated on these videos, they built a set of models that allow them to reconstruct visual imagery from brain response.

The reconstructed images are built from a library of 18 million seconds of YouTube video. A Bayesian system matches the brain signals to 100 likely videos and averages those videos into a visualization of what the scientists saw. We see the video the experimenters watched, then the reconstruction made from the brain signals and the collection of YouTube clips.

In other words, the scientists are using fMRI to pull images they’ve seen out of their brains and put them onto screens. That’s a pretty mind-blowing idea. And if you read the paper, you’ll see that the researchers aren’t just thinking about reconstructing images from sight: “This is a critical step toward obtaining reconstructions of internal states such as imagery, dreams and so on.”

The paper led Bob Moon on Marketplace last night to declare that scientists were exploring concepts he’d never even thought about.

It’s something I’d thought about, but only because Wim Wenders had thought about it at length.

“Until the End of the World” is one of the strangest and most beautiful movies ever made. It’s a deeply flawed gem, a commercial disaster remembered more for its soundtrack than its narrative. That’s in part because the film that screened, at over two and a half hours, was the “Reader’s Digest” version of a 280 minute film that Wenders shot and realized he could never get shown in theaters. For years, the only way to see the full film was to catch one of Wenders’s screenings at universities. (You can now purchase a set of region 2 (European) DVDs, which won’t play on a US DVD player, but which will let you see the full film on a laptop or other player.)

In an odd way, the 280 minute version is, itself, an abbreviation of Wenders’s vision. Wenders is a master of the road film, and Until the End of the World wanders from Europe to North America to China to Australia, and eventually into orbit above the planet. But Wenders had hoped to cover all continents, and the film was originally intended to end in the Congo, echoing a musical motif of pygmy lullabies introduced in the first act of the film. I’ve seen the short cut of the film dozens of times, the director’s cut twice, and feel like I’m starting to be able to imagine the film that might have been, had not money, logistics and sanity intervened.

The plot is convoluted, but at its core, it’s about a technology invented by reclusive scientist Henry Farber to allow his wife Edith to see, though she’s been blind since childhood. The camera he invents records both pictures and the brain activity of the videographer – afterwards, the images can be transmitted from the brain of the videographer (Farber’s son, Sam, and his lover Claire) to Edith’s brain, allowing her to see video interviews with her family, who are scattered around the world.

Rather than being exhilarated by the return of her vision, Edith is overwhelmed by how much everyone has aged and the ugliness of the world – she dies shortly after. Claire, who’s proven a better operator of the camera than Edith’s son Sam, becomes addicted to another use of the camera: watching her dreams from the previous night. The third act of the film is dominated by the abstract imagery of Claire’s dreams, reconstructed on film from her neural activity. (The still above is from that section of the film.)

Part of what makes Wenders’s vision of the future so compelling for me is that the technology his scientist creates is far from magical – it requires intense concentration from camera operators to capture images, and Sam briefly goes blind from the strain of capturing images. It’s a profound effort again to transmit those images. In a striking parallel, it sounds like the Berkeley process is a lot of work as well:

“It takes several hours to acquire sufficient data to build an accurate motion-energy encoding model for each subject, and naive subjects find it difficult to stay still and alert for this long. Authors are motivated to be good subjects, to their data are of high quality. These high quality data enabled us to build detailed and accurate models for each individual subject.”

If the authors of this paper haven’t seen Until the End of the World, I can only hope they’ll go out and watch it immediately.

It’s a long way from the visualizations produced by the Berkeley researchers to “the disease of images“, the addiction Claire nearly succumbs to. But it’s fascinating for me to see science catch up with the most speculative of speculative fiction. I can only imagine the debates of ethics – and aesthetics – should we reach a point where we can see each other’s dreams. (I suspect YouTube would suddenly become a whole lot more interesting.)

Fifty years ago, Newton Minow, the 35 year old FCC chairman, gave a speech that’s still studied today. It’s taught in rhetoric courses, tested on the LSAT reading comprehension test and still is invoked in discussions of how communications technology affects entertainment, news, and democracy. The speech challenged broadcasters to actually watch their programming, and urged them to consider whether they were proud of what’s they’d see. It read, in part:

“When television is good, nothing — not the theater, not the magazines or newspapers — nothing is better.
But when television is bad, nothing is worse. I invite each of you to sit down in front of your own television set when your station goes on the air and stay there, for a day, without a book, without a magazine, without a newspaper, without a profit and loss sheet or a rating book to distract you. Keep your eyes glued to that set until the station signs off. I can assure you that what you will observe is a vast wasteland.”

Today, Minow’s daughter, Martha Minow, dean of the Harvard Law School, welcomed her father to the stage at her institution as part of an event titled, “News and Entertainment in the Digital Age: A Vast Wasteland Revisited“. Minow (I’ll refer to Newton Minow throughout the rest of this post) starts his talk by noting that we’re a day past the ten year anniversary of 9/11, a time at which there was no YouTube, no Twitter, none of the social media we discuss today to understand the tragic events of the day. If that shift is difficult to comprehend, it’s much harder to understand the landscape of fifty years ago, when phone calls traveled by wire, when there were no computers, one phone company and two and a half television companies. There was no public television or radio. Audiences, Minow reminds us, were passive – they gathered around the single set in the house and watched in silence.

When Minow came to the FCC, it was a group wracked by scandal – previous commissioners had been fired for corruption. Minow’s relationship was a highly personal one with President Kennedy. He recalls a meeting with Kennedy and Commander Alan Shepard, recently returned from the first American voyage into space. Kennedy was enroute to a speech at the National Association of Broadcasters, and asked Minow what he thought Kennedy should say to the broadcasters. He told him, “Mr. President, tell them that this is the difference between a free and a closed society: when the Soviets send people into space, we don’t know whether they succeed or fail. In the US, we let people see and hear what’s going on.”

Kennedy gave a brief speech to the NAB which used Minow’s talking points and got a standing ovation. Minow’s infamous speech didn’t get quite as warm a reception. Minow reminds us that Sherwood Schwartz, producer of the television show Gilligan’s Island, honored him by naming the sinking ship on his show the S.S. Minow.

Why give such an incendiary speech? Television was the dominant medium of the era. The televised Kennedy/Nixon debate had decided the election. But there was little discussion about public interest and public responsibility on the part of broadcasters. Minow’s contribution as an FCC chairman was to try to expand choice – licensing the UHF spectrum, early cable TV systems and satellite television. When Kennedy invited him to visit the space program, Minow observed that satellites were more important to sending a man into space, because they permitted sending ideas into space, and ideas last longer than people. Minow notes that there’s a strong possibility that the recent events of the Arab Spring were a product, in part, of satellite communication.

Both Minow and Kennedy had lived in cities where there was a strong public television statement. They both assumed that public television would spread throughout the country, but there was no public TV in New York, LA or Washington DC. When Minow left the FCC, he went on to serve on the board of governors of the Public Broadcasting Service, and on the Carnegie Foundation, one of the major funders of public broadcasting.

As someone who’s been concerned with public broadcasting for his entire career, Minow tells us that he’s deeply disappointed by the relationship between money and politics. “Politicians need massive amounts of money to buy radio and television ads. They raise money from the public to gain access to something the public owns: the airwaves.” This is an absurdity – the US is one of the few countries in the world that doesn’t provide access to the airwaves to candidates. In the UK and Japan, it’s not possible to buy access to the airwaves. Much of the cost of American campaigning comes from the media.

Minow ends his remarks with praise for his host: “I wish the Berkman Center had existed 50 years ago,” because the issue of the responsibilities of broadcasters was neglected 50 years ago, and is still neglected today.

Anne Marie Lipinski, the new curator of the Niemann Center, is one of the three designated “respondents” to Minow’s remarks. She suggests that the most inspiring aspect of Minow’s remarks is the idea that we can do better – as individuals, as broadcasters. One of the challenges in helping us become better is defining the public interest. “I don’t think we have a shared ethos around te public interest in contemporary society.”

Journalist Jonathan Alter reminds us that Minow is also the father of the televised presidential debate. While we still see this important form of civic programming, most of what passes for civic discourse online is extremely poor. “The news business is the only business recognized by the Constitution and it’s largely dysfunctional.” Talk is cheap and reporting expensive, he argues – “the vast wasteland has a Tower of Babel on top of it.” Much of the news we get is “people like me babbling on MSNBC or Fox”, rather than the sort of expensive newsgathering required to report facts on the ground.

Yochai Benkler calls on a section of Minow’s speech where he challenges broadcasters to challenge their sponsors: “Tell your sponsors to be less concerned with cost per thousands and more concerned with understanding per millions.” This section points to the core tension between an American broadcast model that is anchored in markets, and the challenges of public responsibility. Public funding for media and nonprofit models tend to be foreign to American audiences. Yet there’s evidence that networks like the BBC produce some of the highest quality news content available.

Benkler provokes Alter by suggesting that there’s the possibility of producing key and investigative reporting via radically distributed methods. He suggests that the Neda Aga Soltan video, which Alter alluded to in his remarks, was an example of the power of citizen production. He (generously) references a talk I gave the week before about the complex interaction of Tunisians on the ground, activists in the diaspora and Al Jazeera – a state-funded media network – to amplify voices in Sidi Bouzid leading to the Tunisian revolution. “Because we all now carry sound, video and text generating and disseminating tools – phones – we’ve got an unprecedented opportunity to close the gap between what costs a great deal of money and what we all need as citizens.”

Lipinski asks whether anyone is prepared to pay for this sort of crowd-sourced media, asking if any of us pay people whose blogs and twitter feeds we read. Minow suggests that this may be the wrong place to ask for support. He notes that the Japanese closely studied media models around the world before starting NHK and based their model on the BBC, including charging a license fee for television sets. “Other countries started building public media before they built commercial. We tacked on public broadcasting after the fact, without a way to pay for it.” This leaves us with a difficult choice: “Do you want the market to decide and provide everything? And if the market is not going to provide everything, do you want to build an alternative system?”

Alter suggests we don’t hold our breath waiting for the rise of a new public media system in the US. What’s happening instead is the fragmentation of what media exists. He points to the evening entertainment market, where big shows like Leno’s and Letterman’s are ceding ground to the Colbert Report. “It’s a move towards greater choice.” But the downside of this move is that we may be seeing a divide between elites who have access to a vast selection of media, and masses who get little critical media. “The political conversation involves a maximum of 10 to 15 million people,” he asserts, “but 130 million vote in Presidential elections.”

Ellen Goodman offers a nutritional analogy. “People don’t want to eat their broccoli, but they still might vote.” She’s suspicious of the idea that public media will produce the broccoli and be able to get people to eat it, because “public broadcasting in the US is weak and designed to be weak.” Proposals that are unrealistic but still worth making for the production of marketing of broccoli might not be directed to our existing public media institutions, she argues, because these institutions may not be capable of innovation. “It’s reasonable to ask these actors to solve our problems, but they are not going to solve them.”

Virginia Heffernan, cultural critic for the New York Times, suggests we consider not just news. When we look at television entertainment, especially HBO and Bravo, we’re no longer facing a vast wasteland. Minow invites us to imagine the forces of art, daring and imagination unleashed on the television screen, and the artistic explosion we’ve seen the last few years suggests that “television both as an art form and a public health hazard makes these things possible.”

She offers a caution to Alter’s skepticism about digital media and direct sources – we quickly found dangerous media online, like Loose Change, a video that offered the conspiracy theory that 9/11 was an inside job. But we also were able to find video of Saddam Hussein’s execution, shot and distributed by an American serviceman. “Our million dollar Baghdad bureau didn’t get the execution story right” because they were working from eyewitness testimony from individuals in the room, and that testimony wasn’t correct. The actual account of Hussein’s final words came from the video, not the reporting.

What’s key in this world of internet video, she offers, is contextualization. As the New York Times invests in international reporting, they need to make a major investment in contextualizing these images and videos. Asked by Jonathan Zittrain, our moderator, how we might take on Minow’s challenge to “do better”, Heffernan asks us to “register as a Wikipedia editor today. Twice, if you’re a woman.”

Zittrain observes that the phenomenon of Doris Kearns Goodwin, sitting next to Heffernan, registering on Wikipedia could lead to some interesting edit battles over Lincoln’s biography. Asked whether she will register as a Wikipedian, Goodwin offers, “I didn’t know I could!” (Note to Jimmy Wales – we still have work to do.) C

With three former FCC chairs in the room, Susan Crawford – introduced as a “shadow FCC commissioner” in the Obama administration – is offered the first FCC response to Minow provocations with a line about “beauty before age”. She responds to Reed Hundt with a quip about pearls before swine(!) and suggests we think about parallels between Minow’s speech in the service of a “handsome young president, with a beautiful family” and suggests that such a speech would be unthinkable nowadays. For one thing, Minow would have been speaking to the wrong people. Distribution networks are now so much more powerful than content providers, and players like Comcast now control programming and internet access. “There’s only four actors in America who have any power” around these issues of content of the media, “and they really believe that personal preferences equal good programming.”

Kevin Martin, FCC chair under George W. Bush, focused his observations on a topic dear to my heart – the state of international media. He observed that business network Bloomberg now devotes significantly more resources to overseas coverage than the New York Times. (For the record, so does the Wall Street Journal – business papers cover international more thoroughly than “general interest” sources…) Despite those coverage resources, some Bloomberg channels have had difficulty gaining carriage on some cable systems, where they are perceived as specialist content.

Reed Hundt, who chaired the FCC under President Clinton, calls his moves to force broadcasters to show three hours a week of children’s programming his way of honoring Minow’s legacy. “Mandating children’s programming turns out to be a violation of the first amendment, to my amazement.” Like Minow, Hundt was “honored” by broadcasters’ response to his work – the WB network’s show Animaniacs introduced a clown named Reed Blunt… and offered the show as evidence of their compliance with creating children’s programming.

Minow points out that lawyers end up as chairmen of the FCC because “it’s the only government agency that’s regulating a medium of communication.” Lawyers who understand the first amendment understand how treacherous it is and how complicated regulation in the space can be.

Asked to comment on Minow’s legacy, Nicholas Negroponte offers the observation that photography is a medium where artists have been the technical innovators, while broadcasting is a field where the engineers have worked out the tech while the artists were creative. What the Media Lab tries to do, he tells us, is do for computer media what photographers have done – advance the field by advancing both the tools and the creativity.

Zittrain invites Minow to comment on the rise of Twitter: “threat or menace?” Minow demurs, arguing “the more communication the better.” And he thanks us for considering these issues of public interest fifty years after he raised their importance.

Terry Fisher offers a summation that introduces several new, important ideas. New technologies, and some of the practices that surround them (though are not dictated by them) are eroding some existing, long-standing dichotomies: public/private, professional/amateur, speaker/audience, news/entertainment, university/society. There are huge benefits and costs to this corrosion. We see the collapse of oligarchies, address of systematic biases, democratization of processes. But we also have fragmentation, loss of a coherent single culture, the rise of a tower of pundit babel, and the superficiality of much programming. This move, he argues, is impossible to stop. Instead, we need to think through the new opportunities the shift presents: the ability to change who contributes to this process. And we need to figure out how to ameliorate the costs we suffer. That means creating distributed models for sifting, curating, organizing, like Wikipedia, Slashdot and academic projects like Jeffrey Schapp’s Digital Humanities project. In this new world, the FCC may not be the prime mover – the real power is located in intermediaries like Google, and if we were to push for the public interest, that’s where we’d apply leverage.