In a post last week about the Open Translation Tools summit in Amsterdam, I mentioned a “book sprint” that was working to put together a book on Open Translation.
Well, they did it. It was released today, and it’s a damned fine piece of work. (I say that independent of the fact that they used my Polyglot Internet essay as the introduction to the book!)

In five days, a team led by the indefatigable Adam Hyde put together the definitive starting point for people who want to learn what Open Translation is, what tools open translation communities use, what models are working for translation communities, and what the unsolved problems are in the field. The book includes case studies of notable translation communities, including Global Voices, Meedan and Wikipedia, as well as extensive lists of tools useful for localization and translation. It’s available, for free, both as a website and a printable PDF, and will both be published as a paper book, and continue to evolve as a project you can register for and contribute to. (It’s licensed under the GPL version 2.)

As with earlier book sprints, the project demonstrates that it’s possible to make a good stab at a guide to a field of work if you’ve got the right people willing to assemble in a room for five days. The first book sprint was instigated by my dear friend Tomas Krag, who got sick of spending all his time on the road in developing nations teaching people about wireless networking. He knew he’d never write a book by himself, so he held a book sprint, based on the idea of a code sprint, at the annual gathering of the developing world wireless community. Participants spent a long, difficult day arguing over the structure of the book, then went to their respective corners to write, edit, repurpose and recycle content from around the web into a comprehensive guide. The model worked well enough that Adam Hyde from FLOSS Manuals adopted it and has used it as a strategy for building new books around conferences.

I’m off to the Aspen Ideas Festival tomorrow, which looks exciting, celebrity-studded, and worth my careful blogging. But I seriously doubt that a team of smart and crazy people will get a useful book out of it, at least not in five days.

If you want to know what people around the world are thinking and feeling, you need help from a translator. Recent events in Iran are a reminder that the internet and citizen media aren’t enough to give us access to events throughout the world - we need tools and strategies for bridging language gaps as well, or we limit ourselves to only the voices we can understand.

For those of us who think the Internet is a powerful tool for international understanding, language is a challenge we need to confront, a complex set of problems we need to address. I just had the chance to join a small band of people dedicated to solving these problems, joining in the Open Translation Tools summit, held this week in Amsterdam. I came away hopeful, sobered by the size and complexity of the problems, but thrilled that such a smart, creative and global group was willing to take on these challenges.

The internet has been polyglot since early days, but the rise of read/write technologies has brought issues of linguistic diversity to the fore. In our experience with Global Voices, we saw lots of people blogging in English as a second language until there were lots of their fellow speakers online… then we saw lots more bloggers in local languages. Once you’ve got an audience that speaks your language, it makes sense to blog, twitter or otherwise publish in that language. It’s extremely difficult to accurately estimate how many people are blogging in Chinese - figures from companies like Spinn3r or Technorati aren’t counting most of the China-hosted blogging platforms. The number is somewhere between enormous and freaking huge, and people who want to know what what Chinese netizens are thinking better hope we figure out how to clone Roland Soong sometime soon. (Roland and the EastSouthWestNorth blog are so important to English/Chinese dialog that I know of several folks who refer to plans for massive Chinese/English translation as “the distributed Roland Soong problem”.)

Other languages are moving online as a way to ensure their survival in a digital age. The 27,000+ articles in the Lëtzebuergesch wikipedia don’t reflect the size of the language (spoken by roughly 390,000 people in Luxembourg) but the passion of that community to ensure the language exists in the 21st century. While Jay Walker may predict the rise of English as the globe’s second language, I’m predicting that the internet will make it easier to document, share and keep alive the world’s linguistic diversity. (They’re not incompatible ideas, BTW, though I still think Jay’s overstating the trend.)

In other words, every single day, there’s more content online in languages you don’t speak, and you can read a smaller percentage of the internet. It’s not just a matter of learning Chinese, though that would be a great first step. We’re seeing content in Tagalog, in Malagasy, in Hindi, and it’s not clear how we’re going to read, index, search, amplify and understand all of it.

The folks at the Open Translation Tools summit (OTT09) have been working on this problem for a long time. Allen Gunn - “Gunner” to anyone who knows him - characterized the participants as toolbuilders, translators, and publishers. But the common ground is that the people represented at the gathering are pioneers, people who’ve pushed the boundaries to ensure that languages can be present online, and that we can translate between them.

Some of the folks in the crowd, like Javier Solá, can claim credit for bringing whole languages online. (That Solá, a Spaniard, can claim that credit for Khmer is its own wonderful story.) Dwayne Bailey, who’s done excellent work bringing African languages online through his project, translate.org.za, reminded the crowd of the painstaking steps necessary to bring a language online: one or more fonts to represent the character set, a keyboard map to allow text entry, appropriate unicode representations, support for the language within software like OpenOffice, the creation of utilities like spellcheckers. Internationalization is now part of virtually any open source project, but it still tends to be an afterthought, and several groups at the summit were focused on the painstaking work necessary to bring Indian, Central Asian and African languages online for the first time.

Thanks in part to the Global Voices tendency to occupy other people’s conferences - we don’t have an office, so we simply send a dozen people to cool conferences and hold our meetings before or after - publishers were probably the best represented group at the meeting. Many of the projects I most admire were represented, including Meedan, which bridges between Arabic and English speakers via translation, and Yeeyan, which translates English-language content into Chinese. It’s interesting to see the different models emerging around social translation. Meedan translates everything, first with machine translation, and then with volunteer human translators, to make English/Arabic conversation seamless. Yeeyan invites readers to suggest English-language content they think Chinese readers would benefit from reading - Jiamin Zhao, who leads their Beijing team, says this hasn’t been very popular with their users, and that much of the translation happens around large, established projects like the translation of The Guardian. And Global Voices just lets anything go - each language team gets to pick what content they want to translate and what tools they want to use.

Some of the publishers are toolbuilders as well. Ed Zad showed off dotsub’s lovely platform for subtitling and translating online video. While dotsub hosts thousands of subtitled videos, many of us know it better as the toolkit underlying TED’s ambitious open translation project. This model of hosting subtitled and translated videos for third parties is a major part of dotsub’s business model - Ed shows us subtitled videos from the US Army, allowing the Army to meet legal obligations to make all their content available to the hearing impaired, at lower costs as dotsub’s tools are far more efficient than other technologies available.

Meedan offers a beautiful set of tools to allow volunteer translators to turn machine translations into more readable, human translations, and is working closely with Brian McConnell’s WorldWide Lexicon, which focuses on giving publishers a great deal of control over how their site is translated while embracing the model of social translation. I was excited to get a peek at Traduxio, which is focusing on translating cultural texts, like Balzac and Tchekhov and building complex translation memories in the process.

One of the central questions at the meeting was whether toolbuilders were building the right tools for translators to use. A number of projects focused on building open source translation memories. These are tools that keep track of how a translator has rendered a particular word or phrase in the past and prompts her with past translations in a new document. Many professional translators use Trados, though it’s apparently one of these tools that’s industry standard, though not well-loved. (One of the odd quirks of the translation industry, Ed Zad tells us, is that translation clients own the contents of these translation memories, not the translators.) It’s not clear whether social translation projects are really using translation memories. We’ve talked about the subject a great deal within Global Voices, but none of our translation teams is using one… perhaps because they’re not aware of open source ones available, perhaps because few of those open source ones are very good, or perhaps because it’s not how they’re used to working. Ziamin from Yeeyan made the same confession - perhaps because we’re working with volunteers who are translating, rather than translators who are volunteering their time, there’s not much push from within our communities for translation memory tools.

There might be more traction for tools that helped with translation workflow. Professional translators tend to be closely project-managed, and work in teams, with a translator, an editor and a proofreader. Most of the social translation models use less complex systems - an editor usually reviews a translated text in a Global Voices community, for instance, but the system isn’t as formalized. And there seemed to be great demand for tools that matched potential readers of texts with translators, systems that could allow readers to flag a text they wanted to read in another language or show translators potential readership for a particular text. I moderated a session on “demand” which generated a wide range of ideas, from seeking data from Google Translate on what documents were most requested by users to creating Firefox plugins that automatically translated texts and allowed readers to request human-translated versions. My Global Voices comrades were exploring a set of ideas about rewarding translators, with recognition, with karma ratings that might translate into professional translation work, with micropayments for translations - all these ideas require new tools and working methods.

Google wasn’t present at the conference, but was the unspoken presence in almost every session. While there was widespread agreement that Google’s machine translation tools were far from perfect - and sometimes farcically bad - they’ve been getting lots better and some participants wondered whether we should be putting the effort into building new social translation systems if they’re going to obviate all our work in a few years. Personally, I think it’s a bad mistake to stop work because we think Google might be working on the same issues.

The languages where Google is good are ones where we’ve got huge corpora - sets of documents that exist in two or more languages, which have been “aligned” by algorithms so that it’s possible to see how one phrase has been translated into another. A corpus like the Europarl Corpora - which contains millions of aligned sentences in eleven languages, taken from human translations of European parliament proceedings - can make it fairly easy to build these tools… though one wonders if they’re better at translating bureacratic memos than casual conversations. (Another major corpus, the Acquis Communautaire, offers the whole body of EU law in 23 languages. Sounds like a blast to read.) These statistical machine translation methods get stronger as we get more aligned documents available.

But some languages don’t have large corpora available - I don’t know where we’re going to find a large set of English/Malagasy translations, for instance. In these cases, rule-based machine translation might work better - one of our participants, who studies rule-based systems, argues that they’ve proved their utility in translating between closely related languages like Spanish and Catalan. They parse sentences into parts of speech, or into more complex intermediate representations, then translate word by word, restructuring the sentences into grammatically correct forms. Our friend pointed to a study he’d helped conduct which saw these rule-based systems doubling the efficiency of human translators from 3000 words a day to 6000 words, in closely-related languages.

My sense is that the most exciting potential in the near future may be to use social translation to create corpora that could benefit statistical machine translation. That probably means ensuring that Google - admired and feared at gatherings like this one - has a seat at the table in a future discussion.

It’s a long path from the discussions in Amsterdam to a system that allows me to stumble upon a blogpost in Persian and request (and perhaps offer a bounty for) a translation. But those conversations have to start somewhere, and it was a pleasure to have a ringside seat for them in Amsterdam.

One of the projects taking place around the OTT summit is a “book sprint“, a five-day project to write a book that outlines the state of the art in open source translation systems. If that sounds crazy… well, it is, but not as nuts as you think. My friend Tomas Krag pioneered the model a few years back with a brilliant book on wireless networking in the developing world, and it’s been adopted by the fine folks at FLOSS Manuals. I’ll link when the book is available… which should be about three days from now!

You can read notes on each of the sessions on the OTT wiki - it’s a great summary of the discussions that took place.

Chris Csikszentmihayli opens the morning’s session at MIT’s Knight News Challenge conference with an overview of his view of the world - “It’s my view from MIT - MIT wouldn’t endorse it, they’ve been quite specific about that,” he quips, a reference to the university’s unfortunate decision not to grant him tenure. Chris is now focusing on managing the Center for Future Civic Media, and outlines one of the most exciting projects, ExtrAct. The project calls attention to the process of natural gas extraction via fracturing, a process that exposes millions of rural Americans to incredibly toxic chemicals. ExtrAct tries not just to document the practices of fracturing, but to help rural, poor, highly disconnected people organize, get media attention and fight some of the harmful effects of these practices.

What do we, as a society want, Chris wonders. A free and just society. Journalism, openness and transparency and democracy have all emerged as means to that end. Technology, leveraged correctly, can sometimes be a means to that end. Sometimes technology is the enemy of a free and just society. Alan Kay famously said, “the best way to predict the future is to invent it.” Some scholars have suggested that tools control what we can do. Yochai Benkler proposes that it’s not just about the tools, but about how we use them. Bruno Latour suggests that “technology is society made durable.”

Last night’s talk, Chris summarizes, was the “rending of garments” about the death of the daily newspaper. He points out that newspapers put another group out of work, “people so dedicated to their work that they took oaths of celibacy.” (He resists the inevitable geek puns.) The press put the monks out of work. But technology isn’t evenly distributed - head to a city in the developing world and you’ll find scribes, often organized around the post office so they can help illiterate people write letters. (I’ve seen scribes in cybercafes in Kigali…) The Media Lab, Chris tells us, makes its money from fear, taking funds fro sponsors who are slowly going out of business, like the recording industry. The implication, I think, is that documenting these changes - and demonstrating their inevitability - is a useful service for helping corporations accept and cope with this change.

To frame the ideas of user innovation and open source software, Chris shows us how “diff” and patching works - the ability to compare two files on a computer system and send the changes between the two. This is the fundamental idea behind the improvability of open source software, and underlies versioning systems like Subversion and Mercurial.

User-driven innovation, as described by Eric Von Hippel, involves motives other than making a profit - users who improve products often just want a specific functionality available to the world. They don’t need to sell it, just to have it be usable. Open source projects are political spaces - they’re like community organizing projects. They need to be optimized to allow lightweight participation and contribution. He shows the structure of Linux versus Mozilla - as Mozilla moved from a commercial product into a community one, the structure had to change so that people could add code without having to learn about thousands of dependencies.

What tools allow uprisings to take place? Chris is interested in SMS and its role in organizing protests in places like the Philippines. “Governments would love it if these tools weren’t around” - that’s why they shut down SMS during elections. But other tools end up being useful, even if they’re less obvious. Planespotting websites allowed researchers to break the CIA torture flights story - the data was never intended to study torture, but it proved useful for another, critical purpose. This leads Chris to emphasize the importance of laws and practices that ensure an open and free press in a digital age. This might mean supporting Open Street Maps instead of Google Maps, so the maps are reusable and reproducible. It might mean supporting edge figures like Richard Stallman - who Chris analogizes to Reverend Elijah Lovejoy, killed in the early 1800s for his support in print for abolition.

Chris closes his talk with remarks on Jean-Jacques Rousseau, who wrote not just political philosphy but “bodice-ripper novels”. These novels allowed individuals to “live in the skin of others”, experience the empathy that comes from living for a while as a servant or a noble. The daily paper, he believes, can give a sense of community empathy, the ability to live another’s experience through storytelling. That’s something we need to preserve and cultivate as we move into a digital future.

It’s been an interesting few days for people who study social media. As the protests over election results have continued in Iran, and Iranian authorities have prevented most mainstream journalists from reporting on events, there’s been a great deal of focus on social media tools, which have become very important for sharing events on the ground in Iran with audiences around the world. I, like many of my friends at the Berkman Center and Global Voices, have spent much of the past two days on the phone with reporters, fielding questions about:

- Whether social media is enabling, causing or otherwise driving the protests in Iran
- How Iranian users are managing to access the internet despite widespread filtering
- The ethics (and practice) of distributed denial of service attacks as a form of information warfare
- Whether such online activities are unprecedented

Rather than tell you what I and colleagues have been saying to reporters, I’ll point you to one of the better stories, by Anne-Marie Corley in MIT’s Technology Review - she interviews several of my Berkman and Open Net Initiative colleagues and outlines the argument many of us are making:

- Social media is probably more important as a tool to share the protests with the rest of the world than it is as an organizing tool on the ground.
- Iranians have been accessing social networking sites and blogging platforms despite years of filtering - there’s a cadre of folks who understand how to get around these blocks and are probably teaching others.
- Because so many Iranians use social media tools - often to talk about topics other than politics - they’re a “latent community” that can come to life and have political influence when events on the ground dictate.

Gaurav Mishra rounds up dozens of blog and MSM articles and offers an excellent overview of arguments around these questions (with a strong dose of his own interpretation, much of which I share.) He references Evgeny Morozov, who’s got a thorough denunciation of DDOS as a strategy for protest, correctly pointing out that it mostly functions to make participants feel better about themselves by giving them a way to feel involved with the protests. Unfortunately, unlike positive online gestures of solidarity (retweeting reports from Iran, turning Twitter or Facebook pictures green), this one does little more than piss off sysadmins, helps Iranian authorities make the case that forces outside Iran are “attacking the country” and encourage user-driven censorship as a response to unwanted speech.

So, given the wealth of commentary on the questions above by folks smarter than me, let me weigh in on some of the questions I haven’t heard asked.

Biases and social media - One of the reasons MSM outlets are so focused on social media is that they’re not able to deploy reporters to cover these protests. In some cases, the majority of reporting from the ground is coming from social media. It’s worth asking what the biases might be in amplifying those social media reports. Ahmedinejad’s supporters tend to be poorer, more rural, less educated and more likely to speak Farsi than Mousavi’s supporters - a picture of the protests via social media runs the danger of overstating Mousavi support or minimizing Ahmedinejad support. We’ve been trying to counterbalance this a bit at Global Voices - Hamid Tehrani, our Iran editor, did a brief roundup last night of bloggers supporting Ahmedinejad. It’s worth noting that the posts he quotes are all in Farsi: language may well be a barrier that is influencing coverage as well, if voices for reform are easily quoted in English and voices for the status quo are in Farsi.

My friend and colleague David Sasaki reminded GV editors that bloggers had predicted a Rafsanjani victory in 2005, and suffered their “Howard Dean” moment when it became clear that their candidate had little support outside the most liberal bloggers. That’s a very different situation than what’s happening now - the hundreds of thousands of peple in the streets points to profound support for Mousavi - but reminds us that the online voices from Iran, especially the English-speaking ones, probably aren’t representative of mainstream opinion.

An Iran story, not a social media story - Iran is one of the countries American and British media pay closest attention to. The use of social media for protest - especially to promote a protest to international audiences - is far from unique. But because there’s such strong media focus on Iran, and such interest in the use of social media for protest, this is a perfect storm for interest in this topic.

I’ve been asking some of the reporters I’ve spoken with where they were on other recent social media and protest stories. Citizen media has emerged as one of the key spaces for journalism in Fiji in the wake of a coup government that’s censoring mainstream media. It’s been a key source of information in Madagascar as that country’s suffered through a violent change of government. (One reporter who I mentioned this to remarked that Madagascar was “just a speck of an island somewhere”. That speck is twice the size of Great Britain and has the population of Australia…) In Guatemala, online media publicized the assasination of a lawyer by forces close to the president… and government authorities began arresting people for twittering the story to amplify it. These weren’t huge stories for most newspapers - the Iran story is huge not because of the social media aspect, but because protests in Iran are a huge story independent of citizen media.

Flock - I’ve written at some length about homophily, the tendency of birds of a feather to flock together. Turns out that reporters flock, too. It’s somewhat amazing to me the extent to which reporters from really good newspapers are all asking the same questions. I’m glad that people are taking a close look at the phenomenon of social media in the Iranian protests - it’s an important, fascinating and worthwhile topic. But there’s a lot of topics out there, and I wonder whether we benefit from a thousand well-researched stories on this phenomenon rather than a hundred, and nine hundred other stories.

MIT’s Center for Future Civic Media is hosting the annual meeting of Knight News Challenge winners at MIT. In typical MIT fashion, we’re given technical goodies to play with - nametags that include a massive QR code and small, flashing devices designed to remind us that we should make sure to mingle and “spread our social capital”. Our host, Chris Csikszentmihalyi, attempts to reassure us that these tools - based in part on the OpenFace toolkit - aren’t designed to surveil us. Right. I’m calling it the Total Csik Awareness System.

While the majority of our conference is a barcamp, with sessions finalized over dinner tonight, the opening plenary is a question and answer, hosted by Henry Jenkins, who’s making one of his final MIT public appearances before moving to USC. He’s interviewing Knight Foundation president Alberto Ibarguen and sociologist/author Eric Klinenberg. The title of the plenary is “News, Nerds and Nabes,” chosen because these three terms came up in all the MIT organizers’ talks with the Knight folks.

Henry asks Alberto whether he, personally, can save the news industry - a reference to a provocatively titled article in Forbes. What, precisely, would we try to save? Alberto explains that the News Challenge was invented to recognize that Knight was “teaching best practices about an industry that was changing radically” - i.e., teaching people the old ways. Part of the change was recognizing that the foundation didn’t know what the answers would be - instead of offering out of date wisdom, they’re sponsoring experiments in neighborhoods, and sometimes in whole nations. The hope is that the solutions will emerge, in part, from this five year experiment.

In the spirit of holding Alberto to his past press statements, Henry asks him whether he still believes that national newspapers aren’t at risk in the current crisis in journalism. He believes the major papers - the New York Times, USA Today and the Wall Street Journal - either have a “different or delayed” set of problems from regional newspapers. Those regional papers are hit very hard, by Craigslist and earlier by Monster.com. The expense of newsgathering at a regional paper, he argues, is much like the cost of newsgathering at a nationa level - the reporter might spend months unearthing a story. Very local papers will survive on little league scores - they’re not threatened in the same way - but these regional papers are in real trouble. He continues, saying that Knight is “neither smart enough or big enough to save the news business.” In part, the focus is on ensuring universal broadband access, if that’s where community discussion is now moving.

Eric talks about his book, Fighting for Air, which describes the consolidation of local media markets and the damage this consolidation has to local media markets. He references the shift in radio from locally produced content to content produced centrally, designed to sound local. “Most innovative media producers are trying to generate original reporting, but the honest ones admit they need the local paper” as a source of basic reporting for them to comment on. In our current economic crisis, money is flowing to state governments and our traditional watchdogs - local newspapers - are weaker than they have been before.

Henry points out that recent studies from Pew suggest that fewer than half of Americans believe that losing their paper would hurt civic life a lot. Less than a quarter of the youngest would miss the local paper, but more than two thirds say they get local reports from tv or online. Eric isn’t surprised by this - he argues that people have become more disenchanted with their local papers as the second and third papers have disappeared in most markets. However, he believes that most people are reading newspaper content, even if they’re reading it online.

Pointing to statistics that suggest that the average American moves six times in their lifetime, often seeking new economic opportunity, Henry wonders whether this lessens people’s interest in local news. Eric questions the statistics, and argues that we’re more rooted today than in the past, and suggests that most of these moves are local and shouldn’t cut down on interest in local news. Alberto suggests that research Knight is conducting suggests that people’s attachment to places is more about the physical features of a community and its cultural attributes. Eric wonders if this is because we’ve lost faith in local institutions, including newspapers.

Henry offers Clay Shirky’s recent essay on the future of news, suggesting that we need to think about the survival of journalism, not about the future of newspapers. He points out that, in Boston, we’ve got two newspapers, but still don’t get very good information about local elections and candidates. What can we imagine filling these essential community functions in the future?

Alberto points to some of the experiments Knight has sponsored like spot.us and Printcasting might introduce models that could help community news in the future. But he suspects that we’re going to see media focused on a “leadership group” that’s willing to pay for high-quality reporting on paper, a media that reaches elites who are willing to pay for it. Eric wonders whether we might see a subset of Americorps - MediaCorps - which could train a future generation of journalists and rebuild a statehouse press corps with “not all that much public money.” (Ibarguen is skeptical of this idea, and wonders whether the public money would be a form of pressure on these reporters.)

Henry references the current unrest in Iran and the focus on citizen media and wonders what lessons we might learn from the current protests. Alberto points to the reactions in Spain to the Madrid bombings. While the government tried to tie the events to Basque separatists, citizen media argued that this was an incorrect interpretation and helped lead mainstream media to investigate and eventually debunk these claims. Eric acknowledges the power of citizen media during crisis situations. The issue, however, is that our infrastructures may not be able to support such heavy use of mobile phones in disaster situations.

(I, unfortunately, had to duck out at this point and talk to a journalist… from a threatened, regional newspaper, as it happened… :-) My sources tell me that the conversation focused on business models, particularly models for the continued survival of print media, or of serious journalism in digital media.)

I’m having a blast at Beyond Broadcast despite fighting off a bad cold. The organizers have done a great job of moving beyond the usual suspects and bringing in people I’m thrilled to listen to, like Nouneh Sarkissian of Internews Armenia, talking about the innovative work her group is doing linking Georgian, Armenian and Azeri youth around media production camps. And I just got to meet a hero of mine, Tonyo Cruz from TXTPower in Manilla.

But I’m also working my ass off. I moderated a session earlier today on “Global Media’s Role in a Digital Era”, which Ivan kindly blogged here, and I’m speaking in today and tomorrow’s wrap-up sessions, which is forcing me to take good notes and put slides together.

And so… not so much blogging from me. Sorry. There’s a pretty good twitter stream. Jessica Clark in particular is kicking ass and taking names. And I’ll post when I’m able.

The opening panel discussion at BeyondBroadcast is titled “Local Perspectives” and it invites citizen media innovators from around the world to show off their work. Unfortunately for the schedule, the panel includes six terrific speakers, roughly twice as many as could fit in the allotted time.

Myoungjoon Kim of MediaAct in Korea, a community media center, tries to explain the unique features of the Korean media climate. Korea has a level of bandwidth that makes the US look pretty pathetic. Actvist media emerged at the same time as Korea reformed along neoliberal lines. Media was deregulated, and there was a recognition that community media couldn’t just include traditional broadcast media, but needed media education, community radio, and community centers that allowed people to create media. The work his organization does offers more than 200 courses to more tha 5000 members who work to create media in a South Korean context. He tells us that for his work to succeed, he’ll need broad alliances, need for reforms in policy structure and increased infrastructure to teach media.

Lova Rakotomalala, Global Voices correspondent for Madagascar, talks about the relationship between citizen media and the political crisis in his come country. 2009 has been extremely trying for Malagasy - the two cyclones that have left thousands homeless have barely made the news. Instead, the little international attention that focuses on Madagascar has focused on a political crisis - public protests which have led to a military takeover. Not only has there been little reporting on the crisis - media companies have been providing divisive propoganda, not helpful reporting.

This situation has led Malagasy to fear democracy - less than 24% of the popular now express enthusiasm for democratic government. There’s widespread resentment towards the international community for perceived meddling in Malagasy affairs. And it’s clear that Madagascar needs a comprehensive agricultural policy.

Lova was one of the founders of FOKO Madagascar - founded in the wake of TED Africa in Arusha by Harinjaka, a prominent Malagasy blogger, the goal of the project was to help Madagascar become more digitally literate and present, and to send the message that Madagascar is “open for business”. Lova quotes Mike Tyson - “Everyone has a plan until you get punched in the mouth.” As the crisis spread in Madagascar, Foko began documenting protests in the street, trying to fill the gap in international reporting.

Citizen media in Madagascar includes not just the FOKO bloggers on the ground, but a network of 55 bloggers living in five countries. They use blogs, Flickr, twitter and SMS to communicate, and their perspectives are aggregated on Rising Voices and Global Voices. By working with Ushahidi and Frontline SMS, the project is able to involve a much broader group than just the 160,000 internet users in Madagascar - it reaches 2.2 million mobile phone users. This work has led to international attention, including stories on CNN and in the Wall Street Journal. This is great, but there’s still only news coming from Antananarivo in mainstream media, while Foko reports from five different cities.

While the internet reaches very few Malagasy, it’s critical for the diaspora, and for the public perception of Madagascar. The current government wants international recognition and has proven willing to intimidate journalists and bloggers - there’s a desperate need for a structure to protect these reporters. But we’re also seeing evidence that social media helps organize social movements, like the movement to free Razily, which ultimately succeeded in releasing the young man who led Madagascar’s “Tiananmen moment.”

Juana Ponce De Leon of the New York Community Media Alliance talks about finding ways to amplify voices that must be heard. Her organization represents 350 weekly and bimonthly populations, representing 90 communities and 50 languages. The organization began as a set of programs for the New York independent press association, but took on special importance in the wake of 9/11, helping bring voices and stories from the Muslim world into the press during a tense and stressful time.

NYCMA doesn’t focus on original reporting - their work is primarily about translation. “It’s a forum for people who make this media” to bring coverage of communities to a wider audience. While the website doesn’t get overwhelming traffic - about 20,000 visits a week - it’s read heavily by NY city and state government agencies.

Ponce De Leon explains that the economic slump has hit her members hard. Little businesses that support community media are having financial problems, and they’re sometimes unable to support local media. There’s a shift from print to internet, but it’s much slower than in mainstream media. Roughly 39% of the organizations she works with have strong, interactive websites. Some are moving directly to internet radio, which is likely to serve as a hub to facilitate connections for diaspora communities.

In the near future, the main focus is on the 2010 census. New York has at least 150 languages represented in the school system - it’s extremely worrisome that the census is being conducted only in seven languages.

Daudi Were, legendary Kenyan blogger, starts his talk with a story about Kenyan prisons. Every ten years or so, Kenya’s prisons explode in violence. Each time, the minister of home affairs is dispatched to the prison to write a study on what’s going on. Daudi tells us that, decades ago, a prisoner tried to hand the minister a letter - he turned away, not acknowledging it, and the prisoner was later beaten. Fast forward to today, Daudi tells us, when some of the ministers had been in prison in the 1980s. They can ignore what’s going on in the prisons, but video ends up being released and news gets out - newsrooms get mobile phone footage of wardens beating prisoners to death.

Digital tools, he tells us, are bringing people into conversations even when people are reluctant to address the issues at hand. Democracy is government by discussion, and Daudi tells us, it’s based around the idea that the other person has something to say that’s worth listening to. Decisionmaking by discussion is very African - if you marry a woman, you may end up spending a long day negotiating her dowry. You could probably complete the debate in ten minutes, but the discussion takes forever because you’re avoiding conflict. That’s what decisionmaking structures like Indabas are about - we have discussions until we can work through most conflicts.

Blogs today create a new space for discussion. “Blogging is probably the most African thing you can do online today. I’m pretty confident that if my grandmother had the internet, she would have been a blogger.”

It’s not content that’s king, Daudi tells us - it’s content and community. This is one of the strenghts of Global Voices, he argues - bloggers discover that there’s a community that has their back. This is also a strongly African idea - “Ubuntu means ‘You are, therefore I am’”. Identity and existence is a function of community.

The rise of new media in Africa is exciting, but it can be very scary. It’s fun to watch the Kenyan government put exam results online and have servers taken down from the load of proud grandparents in Canada logging online to read them. But when Kibaki declared himself the winner of the 2007 elections and began naming ministers, Daudi tells us, the new ministers’ farms were burning before Kibaki finished reading the statement. Violence can spread as well as opinion, information and news. The lesson, Daudi tells us, is that people want to be relevant and want to be heard - if we can’t find ways to let them speak, they’ll burn things instead.

Antonio Cruz introduces himself as being from the country of the country of Manny Pacquiao. If you don’t know who that is, you’re not a boxing fan, but you’ve got something in common with most of the folks in the USC audience. The Phillippines are an enormous country, the 15th most populous, and it’s a country that’s has a huge diaspora and a population scattered over thousands of islands. It should come as no surprise that the country has embraced the mobile phone, with 70 of 90 million residents owning phones.

TXTPower, the organizatio that Cruz helped to found, helps organize citizens and consumers via mobile phones. Huge demonstrations helped topple the previous government and bring President Gloria Arroyo to power… and a clever ringtone campaign almost toppled her. And major consumer movements are organizing against mobile phone tarrifs and taxes.

TXTPower’s methods are pretty funny. To protest a special SMS tax - which would affect the 2 billion SMS sent in the country per day - TXTPower circulated the Speaker of the House’s personal mobile phone number. The thousands of messages received caught attention from the most important local newspaper. In the wake of a fiscal scandal about vote rigging, an audio clip of the President (allegedly) asking a colleague whether an election had been correctly fixed became a hit political ringtone, and TXTPower’s server was taken down by the interest.

TXTPower turns eight years old this August, and “we’re confident of winning more battles.” One of the co-founders (Mong Palatino, the Southeast Asia editor for Global Voices) was just elected to parliament. And new campaigns focus on the costs of mobile phone service, on training people to learn how to get more out of their phones, and on a political campaign to ensure that Arroyo doesn’t turn into “an eternal leade” - actions on are being coordinated on Twitter, Plurk, Facebook and other social media.

Beyond Broadcast 2009, the fourth edition of a conference that focuses on the future of public service media in a digital age, starts today in Los Angeles at the USC Annenberg school. Dean Ernie Wilson notes that it’s a rainy day in the middle of an economic depression, a tough time to get excited about the future of public media. But there’s reasons for hope - as this conference has moved from the Berkman Center to MIT to American University, and now to USC, the issues on the table have shifted. Our focus here is on hyperlocal media and “hyperglobal”, the ability to share ideas across international borders using digital media. As such, Dean Wilson and his team have made an effort to internationalize the conference, bringing speakers and participants from around the world and focusing much of the program on community media across the world, both in American communities and in other nations.

Wilson offers some challenges for the audience:
- How do scholars of media provide insights that are actually useful and informative to local communities?
- How do we build a future that’s more digital and more democratic?
- How do we take advantage of the success that media is having in countries like India, where increasing incomes and education are helping newspapers succeed?
- What’s the role of public media in countries where public-funded media has historically been propoganda?
- How do we maintain public broadcasting in the face of fiscal pressures? And is public media just what we see on PBS? Or what’s on blogs?

Most critically, Wilson offers a call to action: “If we don’t link digital change and democracy, who will do it?”

Dean Wilson introduces Henry Jenkins, who’s appearing for the first time on a USC stage. He tells us that, after 20 years at MIT, he’s hoping to be at USC for the next twenty. Jenkins explains that he’s a highly visual speaker, and often looks for images to inform his speeches. This led him to think about images of democracy. The images we have tend to be pretty retro: pictures of the American revolution or of 1930s popular protests. Jenkins offers the Norman Rockwell painting, “Freedom of Speech”. The figure, a man in a public meeting, nervously standing up to ask a question of authority. “Put this guy in pajamas and we’d recognize him as a blogger.”

How does this image of democracy change if we look at a pink Hello Kitty mobile phone? It reminds us that Rockwell’s picture is all white, almost all male. That the public sphere positied there is public and highly rational, a fixed space and time. This other image juxtaposes a model of citizenship which is more mobile, more mundane, and brings in issues of gender and generational politics.

Jenkins points to Clay Shirky’s recent writings on journalism, quoting him at length: “Society doesn’t need newspapers - what we need is journalism.” We need to shift from saving newspapers to saving society, and that means we’re going to see a blurring of boundaries between professional and amateur media. Jenkins shows us a quote from Morley Safer: “I would trust citizen journalism as much as I would trust citizen surgery.” This is a false understanding of citizen media, a belief that somehow bloggers are driving journalists our of business. “Increasingly, we’re going to see ‘citizen journalism’ as a phrase like ‘horseless carriage’” - it’s useful for understanding the transition, but it’s not how we currently think about cars. We should expect to see hybrid affiliations, like bloggers working with CNN.

But the real problem with the phrase is the idea that journalists aren’t citizens. Or that civic participation can be reduced to journalism. There are new participatory functions, some of which need to be done and have nothing to do with what journalists do. Jenkins invokes the late James W. Carey, former dean of the Columbia Journalism School. Carey considered journalism in terms of two models - transmission and ritual. Journalistic rituals can shape our feelings, make us feel connected to other citizens. This notion of connection with other citizens informs almost every model we have of public spheres, from Benedict Anderson’s newspapers, to Habermas’s coffee shops, McLuhan’s global village and Putnam’s bowling league.

Jenkins defines citizen media as “any use of technology for the purpose of increasing civic awareness and engagement, enabling the exchange of meaninful information,” increasing social connectivity and enabling a wide range of responses to problems. He invokes Jessica Clark of American University, who defines “pubic media 2.0 as the ability to generate publics around problems.”

This idea raises questions about fragmentation - do we have one public sphere or many? Cass Sunstein is worried that digital media will break down a single public sphere. Jenkins invokes a former MIT colleague, Dana Cunningham, who worries that the black public sphere in the age of Obama is losing critical institutions while black voice still isn’t fully integrated. The example of the Reverend Wright controversy shows how the porous nature of digital media can make it harder for discussions to remain within a specific sphere.

Jenkins sees civic media as extending to include online games, wondering how we can learn techniques from online gaming and apply them to realworld problems. He talks about a group in Brisbane, Australia, where the process of taking pictures of the city turns into a group that explores and envision the city in new ways.

Fan communities are finding ways to mobilize as activists. The fans of the TV show, Chuck, rallied behind their cancelled show and organized a “buycott”, going to Subway (the show’s main sponsor), buying sandwiches and leaving behind cards that said, “Chuck sent me”. The show survived, and grew. Jenkins points out that the fans needed to bring a public together, educate them, come up with tactics, and deploy them, all very rapidly. We’re seeing the emergence of groups monitoring copyright takedowns on YouTube, and a Harry Potter fanclub that’s organized 100,000 young people as “Dumbledore’s Army” to fight Proposition 8.

He looks at the spread of Susan Boyle’s remarkable performance on Britain’s Got Talent as a study in civic media and civil engagement. 170 million people have watched the video of her performance, roughly three times as many as watched the finale of American Idol. And these sorts of shows are an introduction to democracy in countries like China, where the idea of choosing between multiple candiates is a political novely.

Boyle’s story, Jenkins argues, teaches us that the model of “viral media” is wrong. Yes, her story spread via blogs, twitter, Wikipedia and Facebook. But it wasn’t viral - he forwarded it on to people for specific reasons, not involuntarily. And Susan Boyle became a deeply political figure, meaning something different to mothers or church groups than to karaoke communities and fashion blogs. The spread of this attention, faster than mainstream media could react, is an indication of what to expect in the future: demand will aggregate and dissipate before the mainstream media turns its attention to the phenomenon.

The emergence of these new communities and their new ways of engagement will ultimately be a boon for democracy, Jenkins argues. His project is to document the ways it happens, and his new book is going to focus on these questions.

I gave a talk at the New Museum in New York City on Saturday, along with Omar Wasow, the co-founder of Black Planet. It was good fun - Omar and I are nerds of the same generation, and as he showed slides of his beloved VIC-20, I found myself throwing devil horns in a gesture of 8-bit solidarity.

The talk was… sparsely… attended. Which is to say, the two friends from Global Voices who came to support me represented a third of the paying audience. And while this can be a bit dispiriting, Lova Rakotomalala was kind enough not only to attend, but to videotape the beginning of the discussion, which is now posted on his blog. (For those of you who can’t get enough video of me - and, frankly, you should seek professional help - Saturday’s talk was descended in part from my Berkman talk on mapping infrastructure and flow, available here.) And the conversation Omar, Lova, Solana Larsen and others had afterwards at the museum, and later over drinks, made for an extremely rewarding seminar experience, if not the grand NYC lecture I might have been hoping for.

Anyway… the talk tried to draw a line between the idea that we should study what people actually do online, rather than what we might hope they would do, to the phenomenon of social translation. As I put slides together, I kept coming back to a talk Zhang Lei gave at the China Internet Research conference. He was making the argument that, while Chinese-speaking internet users may now represent the largest group of internet users, there’s probably much less Chinese-language content than English language.

As Lei admitted, we don’t really know that this is true, however. It’s really hard to get an answer to the question, “What percent of the internet is in English? In Chinese?” Near as I can tell, the last time anyone made a serious, rigorous attempt to answer the question was a study by Excite AtHome in 1999, which looked at 600 million webpages and concluded that 72% were in English. (If I’m wrong and missing the obvious, definitive study, please let me know in the comments - would love to know about it.) I haven’t found that study, but it’s referred to in an excellent piece in the American Prospect - “Will the Internet Always Speak English?” - and might be part of the study announced here, which looks at a wide range of research questions around a corpus of Excite data.

If Excite’s estimates were right, and the Internet was 72% English in 1999, it’s probably much less so now - in the past decade, the ITU estimates that internet penetration has almost tripled, and much of this growth has been in countries where English isn’t the predominant language. The rise of read/write web technologies like blogs have made it far easier for people to author content in their native languages. While the NITLE blog census, last updated in 2003, saw English, followed by Catalan (?!), as the dominant languages in the blogosphere, Technorati’s “State of the Live Web” in 2007 saw English as a plurality, roughly tied with Japanese… and I believe Technorati’s methods missed lots of Chinese blogs and undercounted them. Still, search for “What percent of the Internet is in English?” and you’ll probably get directed to a Wikipedia page that tells you that 80% is a widely cited figure, but it could be lower.

Why is this question so hard to answer? Well, it’s worth a close look at how people have tried to answer this in the past. In 1997, the Internet Society and Alis Technologies used a random number generator to find IP addresses. They retrieved whatever content these pages turned up (if any) and ran the text through a language analyzer to determine what language the page was in. This method was pretty cool for 12 years ago, but it would fail pretty badly today - millions of sites in hundreds of different languages live at the handful of IP addresses that represent blogger.com, for instance.

The NITLE blog census tried to solve this by spidering content, much as a search engine does, following links from one weblog to another. This isn’t a bad method for finding weblogs, but your results depend greatly on what blogs you seed the engine with - start with a lot of Catalan-language blogs and you’re going to get links to lots of other Catalan blogs, while if you don’t have any Chinese blogs in your starting set, you probably won’t find any. Basically, to get a fair picture of what languages are represented online, you’d need a pretty good guess at what languages you should be paying attention to and where to good hubs likely to point to other content.

The Excite survey deals with search engine data. Search engines run huge spiders, which capture lots more pages than most researchers can hope for. And engines are tuned to filter out spam pages, nonsense content designed to game search engines, which tend to defeat most automated retrieval methods. But search engines don’t index all content - they can’t spider sites where they need to log in, like Facebook or other authorization-based sites, like bulletin boards. They miss chat rooms, of course, and don’t always get data on fast-moving sites like Twitter. And search engines aren’t generally very generous with their data - most engines have stopped posting figures about how many pages they’re indexing, and I haven’t been able to find one that publishes statistics on the magnitudes of the languages they’re indexing.

In his talk. Lei Zhang outlined a straightforward, probably overly simple, method for comparing English and Chinese content indexed by Google - search for an English term (he suggested “breast cancer”) and its Chinese equivalent, and compare the number of results. In a study on how Google’s PageRank algorithm works across languages, researchers suggest that “http” is a good term for these sorts of comparisons, as it shows up in text of any language that talks about the Internet. So I did a quick study:

Using data on the top ten languages listed by people who speak them as first or second languages (from the 15th edition of Ethnologue, listed on Wikipedia), I looked at how many entries each language Wikipedia contains, and used Google’s ability to search in selected languages to count matches for the term “http”. Hindi and Bengali both have an order of magnitude fewer entries than their linguistic peers, and Google doesn’t allow you to search for either specifically, perhaps because there are too few pages indexed. German is the second-largest Wikipedia and ranks highly in matches for “http”.

The Chinese Wikipedia is comparatively small, ranking 12th, but outpaces all other languages, but English, on the “http” test. If the http test is a valid one - and I have no reason to believe it is - it implies that there are roughly a quarter as many Chinese pages as English ones, and that Chinese outpaces Japanese by almost five to one.

It would be great if Google or other engines would reveal this sort of data - for those of us arguing that we’re entering a world where massive social translation is neccesary, the polyglot internet, it would be awfully helpful to have a sense for whether English is still the majority language on the Internet, and when that fact might change.

I’ve written in the past about my friend and colleague John Kelly’s excellent work visualising connections in different blogospheres. His best known research is on the Persian-language blogosphere, where his analysis of linking behavior showed clusters around liberal and conservative politics, but also around poetry. Subsequent analyses have seen clustering around different factors. Russian blogs appear to cluster around platforms - Livejournal users link primarily to other Livejournal users, and so the Russian “blogosphere” is a mess of disconnected communities. The Arabic blogosphere clusters based on location, rather than on interest - Egyptians tend to link to Egyptians, Saudis to Saudis.

The Chinese internet, Kelly tells us, has a complex and hybrid form. It has aspects of clustering via platform, but there are also “trading zones”, where people group by interest and mix content across platforms. He’s looking at techniques of “attentive clustering”, joining people together based on sites they’re paying attention to, rather than on direct links to one another. The research is in an early state, but it looks like Kelly’s techniques will be able to release some interesting information.

Roger Dingledine of Tor offers some insights into his unique and exciting platform for censorship circumvention and anonymity. He reminds us that it’s free software - you’re encouraged to build your own Tor network, though you might have a hard time replicating the 1500 active relays and 200,000 users he’s got on his network. Tor has the most users in China, followed by the US and Germany.

Tor is now a “real live 501c3″ non-profit organization, and it’s been funded by an amazing variety of organizations: the US Department of Defense, the Electronic Frontier Foundation, Voice of America, Human Rights Watch and Google. Speaking to all these funders requires using different language. “When I talk to Walmart, I talk about communications security. Talking to my family, I mention privacy tools. To the military, it’s ‘traffic-analysis resistant communications networks’. It’s the same tool, but I phrase it in terms of the characteristics people care about.”

All these users, Roger reminds us, are needed to keep the network robust and anonymous. Good cryptography isn’t sufficient to provide anonymity - you need to disguise who’s talking to whom, which means Tor benefits from being a network used by privacy freaks, online gambling fans and human rights activists. “Nobody tries to break crypto anymore - they just do social network analysis, find the hub, then break into your house.” Tor helps with one aspect of this problem - it disguises a great deal of communication between people who could otherwise be linked via traffic analysis. On the other hand, Roger remembers a training he and I gave a few years back, where our clients explained were being surveiled both electronically and in the physical world, with parabolic microphones intercepting conversations. Online security can only take you so far.

Roger notes that groups like Tor can help control the pace of the censorship and circumvention arms race. The more publicity tools get, the more likely they are to get blocked - Roger’s very interested in building a tool that’s useful for Chinese internet users, but not aiming at overthrowing or somehow overcoming the Chinese government, because that’s almost guaranteed to make the tool a target for blocking and censorship.

Zhang Lei, the founder of Chinese translation community Yeeyan, starts his talk with a story about his last name. While Zhang is the world’s most popular last name, it’s generally considered exotic in the US, and most Americans can’t pronounce it correctly (”Jong”, not “Zang” or “Zeng”.) He sees this as an illustration of the difficulties people have in understanding one another when separated by barriers of language.

While 18-20% of world’s internet users are Chinese, it’s unlikely that Chinese is as well represented linguistically on the net. Zhang points out that there’s really no accurate data on what languages are represented online - he references an old and probably bad cite on Wikipedia that suggests that 80% of web content is in English, followed by German and Japanese. If this is true, there’s a massive imbalance between users and the content available to them. A simple experiment confirms this suspicion. A search for “breast cancer” on Google reveals 38 million pages - a search for the Chinese equivalent yields only 6 million, and the quality of content is much lower.

Machine translation isn’t a satisfactory solution. A simple paragraph of text, translated from English into Chinese via cutting edge technology, yield about one third readable text, two thirds gibberish. There’s a ton of content that would be worth translating from English to Chinese, and we’re not going to be able to do it automatically.

Zhang’s project, Yeeyan.com, wants to be “wikipedia for translation”. His community involves 8,000 volunteer translators, who’ve created 40,000 translations. The community includes 80,000 participants, who are able to comment on or improve translations. Perhaps the most exciting new project is a collaboration with The Guardian, to translate the newspaper into Chinese on a regular basis, producing an official, sanctioned edition - this is an interesting contrast to ECOTeam, which translates The Economist via an informal understanding with the publisher.

The motivation for Zheng’s project is to build understanding across gaps of language. He explains that terms can mean something very different, even in translation: “The term ‘conservative’ in relation to economic policy means ‘anti-freemarket, pro-government control’ - the opposite of what it means in the US.” These misunderstandings get in the way of dialog and understanding. In 2008, we saw major understanding gaps built on language gaps, centered on Tibet and Chinese nationalism. “I can’t solve these problems, but I can translate,” Zheng tells us. “Translation is the first step and a must to bridge the divide.”

Isaac Mao has been blogging since 2002, and he’d be the first to tell you that blogging has changed how he sees the world. His work now is on developing a theory called “shareism”, based on the idea that humans are inclined to share with one another, but that cultural barriers have emerged to restrict sharing, and that losses and absences in our society arise, in part, from our failure to share. Isaac sees the hierarchical system of Chinese society, and several thousands of years of history of top-down control, as providing an especially challenging environment for shareism.

Chinese people, he believes, are being separated into two groups - those who are connected and those who are disconnected. Bloggers spend a lot of time sharing, subscribing to other bloggers, and connecting with one another. They have more authentic relationships to one another, he believes, based on their willingness to share and connect. The unconnected are influenced primarily by mainstream media - the connected can influence each other, can access information that’s hidden from the unconnected and circumvent censorship. Ideally, they’ll connect via social media, access important information, and share information with the unconnected people, empowering them. “This could be the hope and the future of the Chinese community.”

It’s not reasonable to posit the elimination of China’s hierarchical systems - it needs to be replaced with something, and Isaac believes the sorts of connection he’s talking about could offer that necessary structure. He sees this change already happening in small ways - communities that have access to alternative media stop being as dependent on highly controlled mainstream media. As attention switches to these new spaces, business and political leaders need to pay attention to these new spaces, as do foreign journalists. He notes that journalists covering China are now paying close attention to bloggers, not just to established media sources.