I’d forgotten just how much fun ETech is. Not only are the talks some of the most creative and innovative you can hear in the tech community, the room full of people is one of the most congenial, smart and funny you’re likely to encounter anywhere. Tim O’Reilly won’t come out and say that it’s his favorite conference, but he’s willing to declare it the most important that his organization puts on.

I was only able to be in San Diego for one of the days of the conference – long enough to catch several excellent talks, but briefly enough that I’m relying on Ryan Singel of Wired to catch talks that I’m very sorry to miss: Larry Lessig’s plans to change congress; Quinn Norton, who’s now thinking about hacking her brain as well as her body; Joel Selanikio’s celebration of the mobile phone as a tool for transforming Africa.

Singel did an excellent job with my talk as well, The Cute Cat Theory of Digital Activism. I was grateful to have the excuse to explore at more length some of the ideas I’ve been writing about for the past year, and was gratified that the talk was well received. There were several requests for me to post the slides – that’s not really realistic, as they were 100MB and rather video-rich – what I’m going to do instead is post my notes, a bunch of links and a few of the slides. This won’t be an accurate picture of what I said – it’s more likely to be a picture of what I meant to say.

Web 1.0 was invented to allow physicists to share research papers.

Web 2.0 was created to allow people to share pictures of cute cats.

I had a front-row seat for this transition, working with Tripod. We sincerely believed that the purpose of the web was to give college graduates helpful information about renting apartments, applying for jobs and investing their money. Our users rapidly told us that what the web was really about was publishing their own information… which left us with the difficult challenge of figuring out how to make money off of people’s collections of cat pictures.

User-generated content, on average, is a lot less interesting than professional content. But there are a lot more people creating their own content for fun than those doing so for a living, and in aggregate, that content is at least as interesting.

Based on my Tripod experience, I’d offer the hypothesis that any sufficiently advanced read/write technology will get used for two purposes: pornography and activism. Porn is a weak test for the success of participatory media – it’s like tapping a mike and asking, “Is it on?” If you’re not getting porn in your system, it doesn’t work. Activism is a stronger test – if activists are using your tools, it’s a pretty good indication that your tools are useful and usable.

In late 1996, we noticed that Tripod was receiving a great deal of traffic from Malaysia. Searching through the server logs, we found lots of pages in Bahasa Malay talking about “Reformasi” and “Anwar Ibrahim”. I had to visit the Political Science department at Williams College to figure out that we were apparently hosting much of the Malaysian opposition political movement, dedicated to helping deposed and imprisoned deputy Prime Minister Anwar Ibrahim return to power. Malaysian media was largely closed to opposition voices, but investment in internet infrastructure meant that the opposition was able to access the internet and publish material that couldn’t be disseminated any other way. (Several of these pages still exist on Tripod.)

A more economically rational company would have likely removed the Malaysian content, as we had no way of selling ads to Malaysian advertisers. Economic rationality was never Tripod’s strong suit, and we ended up sponsoring Malaysia’s olympic team instead. (They took the silver in Men’s team badminton.)

With web 2.0, we’ve embarced the idea that people are going to share pictures of their cats, and now we build sophisticated tools to make that easier to do. as a result, we’re creating a wealth of tech that’s extremely helpful for activists. There are twin revolutions going on – the ease of creating content and the ease of sharing it with local and global audiences.

There’s been understandable excitement about use of online video by the Obama campaign. I was in Doha, Qatar, when Larry Lessig showed the above video as an example of the way that remix culture could reinvigorate American political culture. Others have pointed to the video as an example of “user-generated swiftboating“, and the potential for amateur nastiness to be even more evil than our debased professional political culture.

I was sitting next to Tunisian activist Sami ben Gharbia at the meeting in Doha, and he nudged me, saying, “We did this years ago in Tunisia.” I thought he meant the idea of using video to motivate voters. Actually, he meant that Tunisian activists – specifically a friend of his who works under the name “Astrubal” had remixed the 1984 Apple ad for political ends. (See my post “Democrats Invent the Remix, only three years after the Tunisians” for more on this story.)

In the Tunisian video, the guy on the screen is Ben Ali, a major opponent of free speech and a long-serving dictator. No matter how negatively you feel about Hillary, he’s a more Orwellian figure, in part because he’s so skilled at PR. Tunisia is more repressive than many of its Middle Eastern neighbors, but it enjoys widespread tourism and was selected – absurdly – to host the World Summit on the Information Society conference in November 2005. (For more on this absurdity, you may want to refer to my posts from WSIS, perhaps starting with this one.) Because Ben Ali is so good at PR, Sami, Astrubal and others see themselves as an ad agency, making videos designed to embarass the government on an international scale.

One of the most amazing of these videos features the peregrinations of the Tunisian presidential aircraft. You wouldn’t expect to see this jet in Europe very often, as Ben Ali is famous for rarely leaving the country. But Sami and Astrubal used planespotter sites – sites like Airliners.net that allow amateur plane enthusiasts to post their photos – to determine that the President’s jet travels a whole lot more than he does. They He used footage from Google Earth and pictures from the plane spotter sites to make a video that shows the power of the participatory web at its best. (Sami has asked me to make it clear that the Tunisian flight video was solely Astrubal’s work – his function was solely to publicize it, on his blog and in talks given about online activism.)

Their video raises all sorts of ethical questions – is it permissable for the country’s first lady to take the Presidential jet, fueled and crewed on taxpayer dollars, for shopping junkets in Europe? Foreign Policy magazine didn’t think so, and ran an article critiquing the first lady. They also published instructions on how you, too, can become a presidential planespotter.

Sami and Astrubal posted the video on their personal blogs… but as known activists, their blogs have been blocked in Tunisia for years. They also posted it on DailyMotion, a video site popular in the French-speaking world. Shortly after, the Tunisian government blocked access to DailyMotion.

This is a good thing if you’re an activist. Most Tunisians don’t identify as activists and might not be engaged with politics. But, like Americans and Europeans, they’re interested in seeing cute cats being adorable online. When the government blocks DailyMotion, it impacts a much wider swath of Tunisians than those who are politicially active. Cute cats are collateral damage when governments block sites. And even those who could care less about presidential shenanigans are made aware that their government fears online speech so much that they’re willing to censor the millions of banal videos on DailyMotion to block a few political ones.

Blocking banal content on the internet is a self-defeating proposition. It teaches people how to become dissidents – they learn to find and use anonymous proxies, which happens to be a key first step in learning how to blog anonymously. Every time you force a government to block a web 2.0 site – cutting off people’s access to cute cats – you spend political capital. Our job as online advocates is to raise that cost of censorship as high as possible.

So why don’t governments block only he offensive speech? Why would governments be stupid enough to close off these tools entirely? It’s a reasonable question and one that’s an active research topic. One answer is that it’s surprisingly difficult to censor the web well. (Pakistan’s recent shutdown of YouTube shows one remarkably stupid and dangerous way to screw up and overblock web traffic.)

If you want to prevent your users from accessing online content, you’ve got four basic options. You can block keywords, block URLs, pollute your DNS or block IPs. It’s surprisingly hard to block keywords – you need to open and examine all the packets crossing your network. China does a bit of this, but mostly blocks keywords within URLs – it’s prohibitively expensive to examine every packet for an entire nation and check against a blocklist. URL blocking simply doesn’t work very well – it’s easy to rewrite a URL and access the same content. DNS blocking is very simple, but it tends to backfire – your smarter users simply switch towards using an unpolluted DNS and you have no way to control their behavior with this technique in the future. And so, most repressive governments block IPs, which limits access to banal as well as sensitive content.

But perhaps this isn’t stupidity on the part of nations. When Pakistan blocks YouTube, it limits traffic to the site. Google notices these sorts of things. Perhaps it’s coincidental that the video named by Pakistan has been removed from YouTube due to a terms of service violation – perhaps not. But while advocates try to raise the price of censorship for governments, smart governments are raising the price for noncompliance for Web 2.0 companies.

My colleagues at the Open Net Initiative began documenting net censorship a bit more than five years ago. At that point, Saudi Arabia and China were censoring widely. Now at least two dozen nations censor the net regularly, and more may be participating in “event-based filtering”, blocking access to political sites before a key election, for instance. My fear, in the medium to long term, is that every nation that constrains freedom of the press will begin filtering the net, realizing that the Internet is where important press takes place these days.

Of course, the activists win sometimes too. When Google Maps became accessible in Bahrain, it let Bahrani activists answer a pressing question in that small, crowded nation – who owns all the land? From the air, it becomes pretty clear that large chunks of the nation are reserved for palaces owned by the royal family.

An anonymous Bahrani activist thought this was pretty interesting, and made a PDF document of screen captures from Google Maps, enhanced with notes comparing crowded communities with spacious palaces. The document flew around the country from mailbox to mailbox. The Bahrani authorities couldn’t block the file – it’s a PDF, and blocking PDFs has nasty consequences for businesspeople. So they blocked Google Maps, which got bloggers like noted free speech advocate Mahmood Al-Youssif up in arms. After a brief block, they simply gave up and let citizens see the site, rather than letting Mahmood and others train people to use proxy sites. (More on this story is available on my blog.)

When governments really want to shut people up, they don’t just block them, they imprison them. Egypt has blocked very few websites – the Muslim Brotherhood site gets blocked occasionally, but most are uncensored. But they’ve jailed Kareem Soliman for his critical remarks about Islam, and they haven’t hesitated to arrest protesters seeking political reform.

This, in turn, has been known to backfire. When Kefaya activist and open-source proponent Alaa Abdel Fateh was one of 700 activists arrested at a protest supporting the independence of the Egyptian judiciary, it was hard for government authorities to know that they were about to have a PR crisis on their hands. Alaa began blogging from prison, passing notes to his wife, Manal, who jointly maintains their blog. These blog posts helped attract international attention to the case, which meant that camera crews from Al Jazeera and CNN covered a situation they normally would have ignored. It probably meant that Alaa spent much more time in jail than he otherwise would have, but it also may have meant that he was safer than if he’d been anonymous in prison.

(A piece of advice I offer at this point in many talks – if you’re planning on being an online activist, marry a blogger. It’s worked very well for me.)

The imprisonment of bloggers has taught activists some interesting lessons about advocacy in the era of Web 2.0. When Global Voices China editor Hao Wu was arrested and detained in Beijing, I and other GV friends wanted to go online immediately and advocate for his release. But that’s not the right way to do things – you’ve got to get permission from the detained person’s family first. And it took Rebecca MacKinnon a month of phonecalls to get his sister, Nina Wu, to agree to let us advocate on Hao’s behalf.

More importantly, Nina began blogging herself. Unsurprisingly, she knew a lot more about her brother than we did, and she wrote much more movingly than we could. Eventually, our campaign focused on translating her posts from Chinese to English and disseminating them as widely as possible. My conclusion from this: good advice for the advocate in a web 2.0 age – “Don’t speak. Point.” (Bruno Giussani explains what I mean by that phrase far more eloquently than I ever have.)

Nina wasn’t a professional activist. She was a successful career woman, a young mother, living the Chinese dream in Shanghai. She became an activist because she was forced to and she reached out for the tools she had access to – which hapened to be MSN spaces. MSN is heavily censored in China – it’s certainly not what we would have chosen for her. But you don’t get to choose the tools – activists use what’s at hand. It’s fine to build tools for activists, but even better to build tools for folks who don’t know they’re activists yet.

(In making this point, I should be very, very careful to point out that I have deep respect for tools that have been developed successfully for activist uses, tools like Martus or FrontlineSMS. My point is simply that there are huge numbers of web users who don’t yet think of themselves as activists who are likely to reach for the tools they have at hand, not to look specifically for tools designed for activists.)

Most activists discover they’re much more effective out of jail. It’s possible that bulk SMS tools – especially Twitter – might be useful in keeping activists out of jail. Alaa now uses Twitter to report on his political activities – this gives friends watching his feed the possibility of relauching the FreeAlaa site, should we see his note that he’s going in to talk to the police, and there’s no message letting us know he’s out of the police station afterwards. (Alaa tells me that tons of people are now subscribing to his Twitter feed and that they should back off because it’s a very boring time right now in Egyptian politics… :-)

Kefaya activists were able to use mobile phone messages, some sent through Twitter, to alert activists to the impending arrest of Malek Moustafa. As activist came to the place where Moustafa was being taken into custody, they attracted a huge crowd of police, who effectively blocked the street and prevented the police car with Moustafa from leaving the street. He was eventually released. Corresponding with Alaa about the situation, he raises questions of whether this was really a victory for Twitter – this is something Egyptian activists have done with SMS for a long time. Twitter may simply be useful in confusing Egyptian authorities, who might choose to block local SMS in a crisis, but might not consider blocking an international SMS number.

Twitter is also becoming more useful in crisis reporting. Viktor Markovic used a Twitter feed to report live on events in Belgrade after Kosovo declared independence; Juliana Rotich has used her feed to report live from Eldoret during post-election violence. And mobile phones are allowing people to report incidents in Kenya and include them within the map on Ushahidi.

Twitter is far from the perfect tool – it’s centralized and easily blocked. But it’s also used for lots of dumb purposes, which means it passes the cute cats test. Lots of the tools that have become most useful to activists have characteristics that un-recommend them for activist uses. Facebook, which has helped organize major protests against the FARC in Colombia, is notoriously bad about letting users pull data out of the system. Imran Jamal spoke about the challenge of trying to move a community of 400,000 users from Facebook to Avaaz, so they could fundraise more easily. (See “Pros and Cons of Facebook Activist“.) One challenge for activists using Web2.0 tools is figuring out when it’s time to get real and get onto dedicated platforms.

What happens when governments begin taking Web2.0 activism seriously? A funny example comes from Belarus. Belarussian leader Alexander Lukashenko noticed that YouTube was beginning to carry a wealth of anti-Lukashenko content, and suggested the Belarussian government might build it’s own YouTube competitor. Belarussian bloggers went one better and built LuNet, a set of parody sites designed to represent a Lukashenko-compliant read/write web. Perhaps the best of the sites was a Google parody – most searches resulted in a page telling you that the KGB was on lunch break and asking you to try again later when they could watch what you were doing. (See Global Voices Advocacy coverage of the story.)

More competent regimes have managed to exert significantly more control. China filters the internet more effectively than any other nation, using a combination of keyword filters, IP blocks and some DNS fiddling. The system is extremely complicated, involving filtering at a national boundary level and throughout the network, with some blocking taking place deep within the national network. China uses some techniques not widely seen elsewhere, including sending RSET packets when certain keywords are detected to knock users offline.

But that’s not the sinister part. Effective as the Great Firewall may be (and, actually, it’s not that effective – lots of dissidents get around it using various proxy techniques), the most relevant Chinese censorship takes place within Chinese Web 2.0 companies – including US companies operating servers in China. There’s an incredible wealth of Web 2.0 startups in China. These companies allow Chinese users to share video, post photos and write blogs. They’re much more useful to the average Chinese user as the tools and content are in Chinese, not English. And, unlike most popular web 2.0 tools, they’re not blocked in China.

And they’ve got censorship baked in. The above image is from research conducted by my colleage Rebecca MacKinnon. She discovered that MSN Spaces, Microsoft’s Chinese-localized and Chinese-hosted service prevented her from putting the terms “democracy” or “human rights” in the title of her blog. According to a report published by RSF, the heads of web companies meet weekly with censors who instruct them on what keywords to block, allowing the system to be extremely flexible and adaptable.

Some Chinese bloggers have responded by being extremely creative in their use of images. Some Chinese bloggers began posting images of river crabs on their blogs. The joke is that the term for “river crab” sounds very similar to the word “harmonize”, a term that had become slang for “censored” – “My blog just got harmonized.” The term “harmonized” became so popular that it became blocked. So Chinese bloggers began to refer to their blogs as having been “river crabbed”. The watches are a pun on “the three represents“, a political philosophy put forward by Jiang Zemin. This is also a commonly blocked term, so has been rewritten as “wears three watches”… which explains the oddly dressed river crab.

Here’s the thing – for the vast majority of Chinese internet users, they’re encountering a much more free information environment than their parents experienced. Michael Anti argues that Chinese society is much freer than the US in terms of personal behavior, especially around premarital sex and homosexuality. The vast majority of young Chinese are enjoying these personal freedoms and are willing to accept a world in which political freedom is somewhat constrained.

China’s censorship genius is that they’ve found a way to let people have their cute cats and have censorship as well. While China will block sites like Human Rights Watch, they won’t block domestic Web 2.0 sites, and hence the collateral damage from blocking banal content doesn’t draw non-activists to become aware of activist issues. Is this unique to China, or will we see this technique spread? It’s hard to imagine Ethiopia, for instance, being capable of building their own Amharic internet applications and blocking all Web 2.0 tools.

It’s also interesting to see what tools China won’t block. GMail, thus far, has remained unblocked – Anti theorizes that it’s popular with the communist party. Skype is unblocked, and it has some intriguing holes in it – Skype voice chatrooms are tailor-made to serve as pirate radio stations. Pipe a podcast into a chat room and you’re broadcasting audio via an encrypted system to users around the world. And China’s unlikely to block MMOGs, even if people periodically stand on hills inside games and shout out the IP addresses of proxy servers.

(Lots more on China and net censorship at “Cute Cat Theory: The China Corollary” and “Michael Anti and the end of the golden age of blogs in China“.)

It seems criminal to give a talk at the ancestral home of Lolgeeks and not talk about the brave and noble Lolcat. We did some informal research within the Global Voices community and discovered that, while our non-north American, non-European colleagues thought Lolcats were very funny, they simply didn’t exist within their own communities. (Trading funny pictures of animals was quite common, just not the leet-speak captioning.)

Our early attempts to propogate lolcats in other cultures have been largely unsuccessul. (That’s a lolcat by Rachel with our cat, Thorn, saying “Oh Hai”…) There’s a real challenge within the world of lolcats – making activism viral probably means making it funny as well as political and heart-wrenching. My single favorite comment on SUP’s acquisition of LiveJournal is a lolcat, which sums up the situation better than any angry post could have.

It’s typical to end these sorts of talks with a call to action, possibly a better one than “export lolcats to repressive nations”. If there’s a single message to the talk, it is that activists are going to use your tools if your tools are any good – watch them, pay attention to them, protect them and learn from them. They’ll make tour tools better, and they’re one of the reasons to make social software in the first place.

I’d expected Violet Blue‘s talk at ETech to be packed. After all, she’s a sex blogger, a professional sex educator, a columnist for the SF Chronicle, and her talk was on the topic of “Sexual Identity Online”. Perhaps it says something about the ETech crowd that competing talks on database structures were as well attended.

This might have been for the best, as Blue’s talk was much more provocative and political than it was tittilating. In essence, it was an extended discourse on the impossibility of online anonymity and the danger of believing that discourse about sex can be anonymous.

She began with an exploration of fetishes, which she defined as “any object, manner of dress, predictament that offers a sexual charge for the participant.” What’s challenging about fetish online is that many fetishes encourage “unintentional participation”. She offers the example of a Flickr group for people with a fetish for seat belts or purse straps that cross between a woman’s breasts – most of the women in this photostream had no idea they were included in this collection as objects of desire. This gets even more provocative as a concept when she shows a collection designed to appeal to people with a fetish for bald women. Some of the women are leukemia patients and certainly hadn’t intended for their images to be objects for desire.

Blue argues that it’s basically impossible not to be sexualized online. “Just join Facebook with a female name and you’ll find yourself getting sexual offers immediately.” Obviously, this comes with the territory when you operate a sex blog. “Why do we sex blog? Because we want to, because we like the feeling, because we’re in love. Because it’s fun. Because we want to help others with expressions of healthy sexuality, help people going through tough problems.” But it can be a real problem to write about sexual experiences when your lovers don’t want to be as public as you do. “Unless we can be truly anonymous, we have to ask our lovers first.” This is a very difficult negotiation, as it crosses all sorts of potentially hurtful public/private lines.

Much of Blue’s talk focuses on trolling. As a columnist on sexual matters for the San Francisco chronicle, she recieves a lot of anonymous attacks online. She thinks about how her sexual identity can either foil or feed the trolls. “My feeling is that playing victim is feeding them, while being ourselves is changing the culture one blogpost at a time.” She found the level of animosity to her Chronicle columns difficult “until the fairy godmothers showed up.” A group mostly of gay men, they sprang to her defense, accusing the trolls of using sockpuppet accounts and generally making fun of them. “The trolls thought they were dropping bombs, but they were pathetic.”

“Anyone managing a social networking website should either have some history with harassment or know someone who’s dealt with online or offline harassment.” Trolls often take advantage of sites where there’s a “trigger-happy profile deletion policy”, destroying the online identity of their victims by reporting them for terms of service violations. Blue specializes in documenting and tracking down these trolls, and reminds us about real world safety as well. “The last time I got a restraining order, the officer said, ‘You ought to get a gun too.’ I already had one.”

She believes we shouldn’t hide online. “Having your identity is your power – that’s why they want to take it away from you.” She maintains files on each troll she’s encountered, and has contributed to wikis that try to document trolling. California now considers online activity to be police evidence in making a harrassment case. She urges you to track your trolls – “they’re easy to find.”

Unfortunately, lots of people who think they’re anonymous can be easy to find online. She tells us about “the Craigslist experiment.” A troll – RFJason – took an ad from Craigslist in which a woman solicited fulfillment of an intense BDSM fantasy, and posted it on Craiglist in another city. He rapidly got over 100 responses, some of which included photos of faces and private parts, and he published all the responses on the Internet, including contact info. The experiment ruined several people’s lives, costing some people relationships and jobs. “The people who responded didn’t make a correct risk assessment in answering that ad.”

Posting an ad like the Craigslist one, she tells us, might be a form of online exhibitionism, pure fantasy. “Posting an ad like the craigslist one might be a lifeline for the person who posted it.” But “ust because someone has a fantasy doesn’t mean they want it to come true in real life.” It’s important to remember this in systems like Second Life as well – people might be engaging in sexual roleplay wearing avatars that don’t share their own gender – this isn’t neccesarily a way of engaging in homosexual impulses – it’s something more complicated… and it can be very disturbing for those attached to having cybersex with people who are the gender they’ve portrayed themselves to be. “Guaranteed gender matters to some – certainly not to all.”

Some bloggers have managed to write about their sex lives and remain anonymous – it requires being very, very careful. She lists Eros blogas an author who has protected her identity for several years. “Girl With a One Track Mind” wasn’t so lucky. She blogged as “Abby Lee” and documented her sexual experiences, eventually publishing a book based on the blog. She was outed in the UK tabloids three days after publishing the book. In her real identity, Zoe Margolis, she lost her job in the film industry, wasn’t made rich by the book, and had the embarrasing experience of having her ex-lovers read precisely what she thought of them.

“We should be participants, never victims. That’s me. I know that to be online is to be sexualized.” In her sex counseling, she talks about “safer sex” – sex is never 100% safe. People make poor choices, but we can educate them to the risks and help them make better ones.

So Blue uses her real name and talks about her blogging policy with all lovers. She now no longer will date people who want their sexual experiences to be kept out of the blog. “My events and experiences are always real. I own who I am. My lovers are real.”

She outlines some of the perils of sex blogging – her article on Wikipedia is vandalized regularly, and she’s received three death threats since she began writing for the Chronicle – all of which she’s posted on her site. The upside – she gets a lot of mail from people who thank her for publishing online, for speaking honestly and frankly about these matters.

Her talk was a bit staged and prepared for my taste, but hey, I suspect that if I stood in front of 200 people and talked about my sex life, I’d have some very careful notes as well. I can’t chalenge her assertions about the sexualization of online life, much as I’d like to – I’m not female, don’t write about sex, and there doesn’t appear to be a widespread fetish for balding, overweight geeks who write about foreign policy. But I would offer a mild caution on the subject of online anonymity. It’s not impossible to be anonymous online – it’s just very difficult to do it right. There are good reasons to try to be completely anonymous online, many of them non-sexual, but it’s a provocative challenge to think about whether one really needs to be hidden when people like Blue are this out in public.

Noah Schactman spoke at ETech just before me – I regret that I’m only posting notes from his talk now, but there’s been thousands of miles of travel and a couple of very long meetings between then and now. Besides which, if you’re really bored, you can go back and catch up on all my TED posts you haven’t read yet.

Schactman wrote an excellent piece in Wired about the role of data networks and personal networks in the US invasion and occupation of Iraq. I assume the article had been given an overly provocative title by a Wired editor – “How Technology Almost Lost the War” – but Schactman’s speech seems subject to some of the same critiques – he may be criticizing technology for some deeply human and strategic failures.

His talk starts by introducing us to Sergeant Joe Colabuno, a US Army soldier who’s so popular with his commanding officers that they’ve basically prevented him from leaving Iraq. He isn’t the best educated guy in the world – before the army, he was managing a steakhouse in Cleveland – and he might not be the most gung-ho – Schactman tells us that Colabuno joined a Psyops team because he thought the Army would let him experiment with LSD. But he’s become indispensible to the success of the Army in Iraq.

To understand why, we need to look at how the Department of Defense has embraced new battlefield technologies. There’s been an obsession with “network centric warfare”, an attempt to optimize the “killchain” much as Walmart has optimized its supply chain. (Walmart was, in fact, the model used to transform the military.) It’s worked pretty well – systems that commanders use see the position of every piece of military hardware, every military contractor, and to graph rocket attacks, IEDs and other enemy actions. Want to plan a safer path for a convoy? No problem – pull up the map and draw a new line that doesn’t pass through the dangerous areas.

This system is called “CPOF” – “Command Post of the Future”. Soliders predictably call it CPORN – “Command Post of the Right Now”. It’s terrific at letting the Army kill folks more quickly, but it does very little to build schools, elect local government, or do any of the other social activites neccesary to stabilize this nation. “Unfortunately, you can’t kill your way to a stable Iraq.”

Asking soliders to focus on rebuilding Iraqi society is “like asking the wallmart greeter to join Cirque du Soleil.” They’re just not qualified, Schactman tells us. “The only Arabic anyone knew was from Team America World Police – they’d drive around in Humvees yelling ‘dirka dirka’ at everyone they passed.” In an 1800-person unit, the only guy who could communicate with Iraqis was a guy who spoke Russian – some of the Iraqi guys had Russian from former military trainers. This obliviousness to local culture meant that guys misread signs on the ground. “Guys would see blag flags waving – they knew what white flags were and assumed these were the opposite. ‘If these guys are flying black flags, they must want us to bring it on.’” The black flags were a Shia religious symbol that was badly misinterpreted by the soliders.

On the other hand, the insurgents knew exactly how local cultures function and how to function within this framework. Schactman points out that the CPOF screens didn’t show where the insurgents were, just where they’d struck before. And they certainly didn’t show the people who might be on the US’s side or might be on the insurgents’ side, depending on specific circumstances on the ground. “Where are the police offers who take money from us and from the insurgents? The shopkeeper caught in the middle? – The dots on the screen need to be grey, not just red and blue.” This is especially true as Iraqi law allows each house to have an AK-47 and a magazine of ammunition – “everyone has the tools of insurgency.” To find insurgents, you need to understand influences and motivations.

This is what Sergeant Colabuna has proved so good at doing. At the start of the war, his unit was responsible for faking out Iraqi troops – they drove around in an humvee with a speaker system that made sounds like tanks advancing or aircraft overhead, designed to make the enemy look in the wrong direction. After the invasion, his job was to spread propaganda through low-tech channels.

His first attempts were pretty unsuccessful – pictures of children killed by insurgents with an admonition, “Why are you letting these terrorists kill your children?” That didn’t go over very well – Iraqis felt shamed, not incensed. His first big hit was a flyer that parodied a local insurgent group, specifying that Allah had reserved places in hell for people who killed without mercy for women and children. That one provoked a split in the local insurgent movement and allowed his colleagues to target and kill some of the insurgent leaders.

What Colabuna had figured out was how to work the local social networks. This, Schactman argues, has been the key to the major successes in stabilizing Iraq. He tells the story of a commander who was having difficulty recruiting local police. He finally discovered that he needed to go to prison and get the blessing of the local tribal leader before anyone would work with him.

A future version of CPOF, he argues, should include information on the social networks in Iraq – tribal and religious boundaries in town, the tensions between them, the key individuals who can shape social movements. But that’s not easy information to get. And there’s a danger in these techiques as well – as Schactman documented in one of his pieces for Wired, Colabuna’s work plays on Shia/Sunni tensions and prejudices. It may have helped to stabilize Faluja, but it might contribute to future civil war.

I enjoyed the talk, but was a bit disappointed that it didn’t stray far from his Wired article. Some in the audience reacted quite negatively – it’s a bit unreasonable, they argued, to blame a failing war on data visualization techniques. Where he and his questioners seemed to be in agreement is that the technology has helped the US military kill more efficiently – as for whether technology can be blamed for killing stupidly and destructively, that’s probably a conversation for a forum other than ETech.

Dr. Paul M. Torrens from the Arizona State University School of Geosciences likes making people panic. Virtual people and virtual panic, that is. He studies crowd behavior, which involves putting virtual people into virtual disaster situations and seeing just what they do.

Crowds, he tells us, are pretty important to understanding social life, to understanding our “ambient social infrastructure.” They are instrumental in history’s most famous events, like storming the bastille. Crowds are complex adaptive systems – their swarming and flocking behavior helps us understand mob behavior, but also how people passing by retail storefronts pay attention to advertisements.

It’s hard to study this stuff in real life. “It’s very difficult to interview a rioter who’s throwing a molotov cocktail. “Excuse me, sir, on on a scale of 1-10, how upset are you right now?” So he builds models – simulations, used as alternative laboratories.

The state of the art in this research is coming from different fields: computer science, physics, movie special effects and urban design. The special effects designers want to find ways to create realistic crowds, composting computer-generated avatars against matte paintings. Physicists are interested in how crowds might serve as models for other adaptive systems, like particles in fluid flow. Urban designers want to know how line of sight moves people through space. And some scientists are interested in building “social force models” that model attraction and repulsion in crowd behavior.

Torrens wants to build scaleable, replicable toolkits to build these models. They’re agent-based – each character has its own motivations and acts autonomously, encountering other actors in the space. It allows modeling of meregence, path dependency and human factors like feedback and lock-in. This might be useful for builing AI models for crowd behavior in games, or for applications like designing better cities or buildings.

These models begin with motion capture data, filming people walking, running, or falling over. You can then use these models to introduce physics into the worlds – “You can try running rioters into steel walls, which is otherwise hard to get past the IRB board.” The motion capture models then get wrapped into textured envelopes, making them look realistic.

One major problem – the tools for building simulations are currently richer than the data sets we have. He’s capable of building very realistic looking and acting people, but modeling crowd behavior requires inferential statistics and other forms of intelligent estimation. Once he’s done this, he can map his virtual crowds into “geospatial exoskeletons” – real-world spaces modeled in code. We see one of these scenes in action: a highly realistic street scene, including young and old people, a few drunks, all intersecting in a crowded pedestrial boulevard.

This is cool, and looks great, but the hardcore stuff is the “extraordinary scenarios”. We watch a group of avatars attempt to find their way out of a crowded building. They’re very bad at it – they cluster in front of a doorway in a pair of wedges. The system works far better when a column is introduced off-center in front of the door. It’s counterintuitive, but it sends shock waves through the crowds to break up these patterns.

A much more complex situation attempts to figure out how police can control a crowd. A model includes aggrieved civilians, rebels trying to instigate rebellion and law enforcement, trying to keep the crowd from spinning out of control. He’s run base scenarios as well as ones that include tear gas, introducing panic in the rioters but allowing police to move unmolested. The idea is to learn from the models and figure out the signs of mob behavior before it breaks out. (This makes me a bit nervous, but not quite as nervous as his next project.) Another project models the city of Salt Lake City and tracks the path of three individuals based on their cellphone signals. Based on their path behavior, we can extrapolate where we might look at building CCTV surveillance tape to identify a set of kidnappers, for instance. (Or political dissidents. Hmmm.)

His work is extremely cool eye-candy with lots of fun implications behind it – I’m looking forward to learning more at Geosimulation.org.

Eric Rodenbeck from Stamen Design shows some of the gorgeous work he and Mike Migurski do in visualizing large sets of data. We take a brief look at pollution maps, commuting time maps in London and then look closely at the Crime Map of Oakland. You can understand very quickly why visualizations like these can be highly political – the city of Oakland briefly shut them down. He argues that the job of visualization is to stand at the intersection, “between analysis and spectacle”.

Chris Melissinos has a great job title – he’s chief gaming officer for Sun Microsystems. He knows you don’t think of Sun as a gaming company, but he demonstrates his gamer cred, with an amazing array of classic video game platforms in his office. He’s a member of “generation pong”, the “bit babies” who grew up on computer gaming and are now teaching their children to game. This means that the drivers of new technology adoption are five to fourteen year olds, the children of these older gamers.

Gaming, he reminds us, is mainstream – it’s a $11 billion market globally, bigger than the recorded music industry. Casual online multiplayer gaming is growing fast, and women over 35 are the fastest growing market segment.

But gaming tech is in the dark ages. “It’s like when Hollywood built mansions for each movie, then burned them down afterwards. Eventually people learned to just paint them.” Gaming tech is driven by game design. It’s not stable, not scaleable, and gets rebuilt all the time. He tells us that 10 thousand users is not massive, that your users don’t care about “shards” – they’re simply pissed off that they’ve paid to use your game and that it isn’t letting them in.

Rather than have game designers become better network engineers, Melissionos would like you to use his tools instead. His group is working on Project Darkstar, a platform designed to support multiplayer gaming independent of game design or platform. It’s java, open source and available under GPL2. It promises massive scale and persistence, and should be easy for designers to plug into, focusing on the creative aspect of their work, rather than on designing networking software.

Sun wants to be your datacenter, maintaining your server room . “Peter Jackson didn’t have his crews design and build their own cameras,” to make the Lord of the Rings films. They see this as a system that empowers small game developers to compete, without needing to raise hundreds of millions of dollars. And they see impications in the business collaboration market with Project Wonderland, a Second-Life like virtual world, based on standards and open code in Java.

Liz Churchill of Yahoo! Research would like some of your attention. But only a little bit of it. She’s interested in information embodied in place, and ways that people can encounter data casually and through serendipity. One of her main projects is the YeTi interface, a system designed to enable communication between offices in Japan and California. It’s an information sharing system based around community screens. Users are more likely to casually encounter information and this information makes it more likely that they’ll have meaningful interactions face to face or online.

You might ask why not use a mailing list or a virtual world – Churchill’s tried them all: The Palace, MOOs, MUDs. They all require a much heavier level of involvement that she’s looking for. She wants to encourage “lecher les vitrines” – windowshopping – and “peripheral participation”. One of the odder features of the system – it watches you interact, and the author of a piece of content can call up short videos of the people who looked at your content. Obviously, this has scaling and privacy concerns. But I can understand why it might work well in an office setting.