Wikipedia is rapidly emerging as the poster child for peer-produced content. While Linux, Apache and other high-quality open source software convinced most in the technical community that “commons based peer production” is one of the most powerful methods to create robust software, it’s been harder for non-geeks to understand the strange and wonderful processes that turn distributed effort into remarkably large and useful works.

Now one million articles strong, Wikipedia is arguably the largest encyclopedia in the world. Its philosophy of radical openness – anyone can add or subtract anything at any time, though changes can be rolled back – hasn’t led to chaos, but a fascinating system that corrects many acts of vandalism within five minutes. It’s a fantastic gateway project for non-technical people interested in contributing to a large, meaningful Open project.

Amazing though it is, Wikipedia is not flawless. It’s got a problem common to almost all peer production projects: people work on what they want to work on. (This “problem” is probably the secret sauce that makes peer production projects work… which is what makes it such a difficult problem to tackle.) Most of the people who work on Wikipedia are white, male technocrats from the US and Europe. They’re especially knowledgeable about certain subjects – technology, science fiction, libertarianism, life in the US/Europe – and tend to write about these subjects. As a result, the resource tends to be extremely deep on technical topics and shallow in other areas. Nigeria’s brilliant author, Chinua Achebe gets a 1582 byte “stub” of an article, while the GSM mobile phone standard gets 16,500 bytes of main entry, with dozens of related articles.

This caught the eye of Wikipedia contributor Xed, who identified this as a systemic, structural bias in the Wikipedia system. He’s launched a project called CROSSBOW – Committee Regarding Overcoming Serious Systemic Bias On Wikipedia – which is looking for ways to address these biases and increase the number of articles on less-covered topics and increase the visibility of the “less travelled” articles that exist.

His comments have led to a lively conversation about what topics are undercovered on Wikipedia and what steps could be taken to get more coverage. Should Wikipedia attempt to provide richer coverage on nations not featured in mainstream media, using traditional printed material to do research, which, in turn, helps correct the Wikipedia bias towards subjects easily researched on the web? (Xed and I have corresponded on this issue, and I’ve suggested my bottom 32 list of Rodney Dangerfield nations, the ones that get no respect…) Should CROSSBOW focus on new articles on African-American authors, undercovered feminists and other topics – in the developed as well as developing world, that have slipped through the cracks, as wikipedia contributor JMabel suggests? Should CROSSBOW focus on translating content from the hundred non-English wikipedias and their 650,000+ articles?

While I think these are all valid directions, I feel the solution to systemic bias in Wikipedia is the same as the solution to systemic bias in open source software development and in the blogosphere: broaden the sphere of producers. Part of what makes Wikipedia great is that people write about subjects they’re knowledgeable and passionate about. I think it’s possible that CROSSBOW participants will become sufficiently knowledgeable about the civil war in Guinea-Bissau that they can write about the conflict… but I bet the article would greatly benefit from the perspective of someone who survived the conflict. I think CROSSBOW is mostly likely to succeed if it can recruit people around the world to participate in the Wikipedia project – in their own languages and in English – to help start filling the blank spots in Wikipedia and helping it reach its full potential.

I’d love to do some work trying to help determine the “holes” in Wikipedia – I’m very interested in thoughts people have on methodology. Does it make sense to hold Wikipedia up against a traditional encyclopedia like Brittanica, and look for areas Wikipedia doesn’t cover? Or does that miss the whole point of a peer-created work?

While I figure that out, I’m starting work on my first Wikipedia article, on conflict diamonds… keep your eyes peeled.

J.D. Lasica, blogger and frequent contributor to USC Annenberg’s excellent Online Journalism Review, emailed me a few days ago with an interesting question: Why does Google News seem appear to have a conservative bias regarding John Kerry?

My initial response was skepticism – Google News is one of my prefered news sources (not to mention something I’ve spent hours and hours scraping data from) and I’d never noticed a clear political bias. But I was able to quickly recreate the effect Lasica was seeing: when you follow the links for “John Kerry” under “In The News” on the right side of the Google News homepage, you get a surprising number of results from “alternative” news sources – highly partisan, bloglike sites that, for the most part, appear to be run by people who will emphatically not be voting for Senator Kerry.

I’ve got a theory on why this happens, which Lasica features in his recent story for OJR – “Balancing Act: How News Portals Serve Up Political Stories” – alongside some thoughts from Google’s chief scientist. Basically, I think it’s an interesting, accidental linguistic artifact which demonstrates just how hard it is to get an AI to do something as complex as laying out a page of news stories. But stop listening to me and go read Lasica’s excellent article.

I posted a couple of weeks back about an experiment I was beginning to run, looking at what headlines from the New York Times get “selectively amplified” by weblogs. With the (generous, much appreciated) help of Kevin Marks, my tools are now checking headlines for their inclusion in Technorati.

Anna, commenting on my blog post, observed that my data would be stronger if I reported what percentage of international stories that appeared in the Times were picked up by bloggers. So I’ve reworked the tools to track the categories the Times filed the stories into, as well as the age of the stories.

Headlines that have appeared on the Times website over the last two days are classified (by the Times web site architects – I’m using the section designation in the story URL) into 17 categories. 10 major categories (arts, business, health, international, national, NY regional, politics, science, sports and technology) have 9 or more stories in them; 7 minor categories (books, dining, education, fashion, obits, travel, theatre) have four or fewer.

Of the major categories, “politics” stories were blogged the most, with 214 links; “technology” lags well behind with 51 posts, followed by “international” and “business” with 43 each. The ranking is a bit different when we consider the number of blogposts per NYT story – “politics” continues to crush the pack with 3.82 posts per story; “technology” and “science” follow at 1.7 and 1.67 posts per story, respectively. “international” falls to 6th of 10 categories, business to 8th. Sorted by the percentage of stories that get blogged, “politics” continues to lead the pack at 78.57% – “health” edges “science” and “technology” for second at 69.23%, and “international” and “business” stay in their places at 6th and 8th.

The categories most ignored by bloggers are sports (0.25 blogposts per story, 15.63% of 32 stories blogged) and New York regional news (0.46 blogposts per story, 18% of 50 stories blogged). There’s a simple regional explanation for this – these sections of the Times are of most interest to New Yorkers (who, though numerous, represent only a small portion of the world’s bloggers), while the other major sections cover national or regional issues.

The reason I’m listing the categories in quotes is because the names are somewhat deceptive. “politics” contains only stories regarding US politics – stories on European politics are under “international”. But many of the “international” stories have a strong US component – of the 14 stories that were blogged, 4 explicitly mention the US, and three, Iraq.

Two focus on Canada, two Europe (France and Greece), one each on Israel and Iran. The only “international” story blogged on a low-development nation is a story about conflicts between white farmers and Masai herders in Kenya. (A quick glance at the unblogged “international” pieces reveals that the Times did run stories on developing nations, including stories on Haiti, Russia, Indonesia and North Korea.)

Here’s the full results, for anyone who’s curious.

I’m planning on running the script for a few more days to see if the results differ meaningfully. I feel pretty confident that most NYT stories that are getting blogged will reach Technorati within 36 hours; searching Technorati for links like “www.nytimes.com/2004/09/23″ gives us a general sense for how many links per day to the Times appear. At 5pm today:

9/23 -> 166 links

9/22 -> 577 links

9/21 -> 594 links

9/20 -> 833 links

9/19 -> 766 links

9/18 -> 310 links

9/17 -> 526 links

I conclude from this that many of the links a story will get will appear the day after a story appears; the variation we see on the 18th probably has to do with the size of the newspaper. So running the script for three or four days will likely ensure that most links to stories will register. I’m also hoping to run an alternative data set (possibly The Guardian) before releasing more complete results in a couple of weeks.

I always have mixed feelings about turning up evidence that supports my pet theory – that bloggers as a whole don’t give more of a damn about the developing world than residents of wealthy nations as a whole. Blogs give us a great opportunity to tell mainstream media what we care about – unfortunately, what we care about is Bush and the iPod. At least I’m converging on a title for the article or book I’m hoping to write on this subject: “The Media Sucks, and It’s Your Fault”.

Early in the life of Geekcorps, I spent a great deal of my fundraising energy talking to large tech companies and trying to persuade them that sending some of their employees to the developing world with us was a wise business move. “You see,” I would say, “the market for IT goods in the developed world is slowing; the critical new markets are in the developing world. Work with us and you’ll have a chance for key employees to get to know those markets and develop products for them.”

This pitch didn’t go over real well. Two companies – Cisco and HP – pointed to the work they were already doing in the developing world as an excellent reason not to work with us. The rest pointed to charitable work they were doing in the US as reasons they couldn’t support “charitable work in non-key markets.” And more than a few looked at me with wide, blank eyes and an expression that read, “I don’t understand your crazy moon language.”

So it’s oddly gratifying to see Business Week’s (international) cover story titled “Tech’s Future” focusing on technology in the developing world. The piece leads with the following: “With affluent markets maturing, tech’s next 1 billion customers will be Chinese, Indian, Brazilian, Thai… In reaching them, the industry will be deeply transformed.” (Many thanks to Rajesh Jain’s indispensibleEmergic for the link.)

Business Week seizes on the general premise I argued for a few years back, pointing out that computer markets in the developed world are expected to grow at roughly 6% annually, while in the developing world, they’re expected to grow at 11% per annum. These numbers are a wakeup call for Dell, IBM and others, who are now scrambling to develop products for these new markets, hiring anthropologists to help them understand PC usage in households around the world and learning what does and doesn’t work in marketing and retailing. (Dell’s build-on-demand model? Not very popular in China, where there’s a premium on seeing and touching the goods one is buying. Dell’s got a tiny fraction of market share compared to China’s big two computer retailers…)

What’s most interesting to me in the Business Week article is the speculation that giant multinationals may lose to culturally sensitive local companies in the battle for these new markets. (This isn’t a new development – there’s a great story about Ghanaian software developer SOFT’s victory over Microsoft in the West African accounting software market reported last year by BBC…) Many of the countries most interesting to multinational tech companies are technical powerhouses in their own right: India has a strong technical services industry; Russia and Eastern Europe have strong security and crypto specialists; Brazil has a great crop of open source hackers; and China has expertise building cheap hardware.

While it’s not surprising that a Chinese hardware manufacturer might be able to go toe-to-toe with Cisco in China, it’s a little surprising (and, to me, very encouraging) to see one go after other developing markets aggresively. China’s Huawei has a 16% share of the router market in China, second only to Cisco – their web site makes clear their global ambition, with maps of their presence in Eastern Europe and press releases about 3G phone service in Africa.

So, emboldened by Business Week, I offer the following “radical” prediction: Over the next ten years, the most interesting developments in IT will come from developing nations, not from Silicon Valley. In the same way that university-student net geeks became critical players in the dot.com boom (because they understood the Internet on a deep level and saw possibilities that others missed), creative engineers from the developing world will become critical players in the next IT boom cycle. It will be interesting to see whether smart venture capitalists “get” that they need to start hanging out in Mumbai or Sao Paolo, or whether they’ll cling to an antiquated model where the US innovates and the rest of the world follows.

Creative Commons has released a “developing nations” license. It’s a simple solution to a complex problem – how does a publisher monetize her content in nations where it’s reasonable to sell it for a profit, but offer it with fewer restrictions in a market where she won’t see measurable revenues for many years to come? The new license allows a creator to license her works “attribution only” to people in developing nations. The legalese behind the license defines developing nations as follows:

“Developing Nation” means any nation that is not classified as a “high-income enconomy” by the World Bank.

(I assume that “enconomy” is a typo and will be fixed in a future revision. That, or I’m really behind in recent developments in the field of enconomics and should make an effort to catch up.)

My trusty World Bank Development Indicators 2002 tells me that only 52 nations are classified as “high-income”. That means the license allows non-royalty use of content in such poor countries as Saudia Arabia, South Korea and Argentina. Overall, the license applies to countries that house at least 5 billion of the people on the planet.

But I’m quibbling. And quibbling aside, this is a very good thing. I think it will have less of an impact on individuals choosing whether or not to release content under CC licenses as it will on large publishers. I’ve spoken to a number of publishers in the tech industry who realize that they’re unlikely to make money off the Armenian or Mongolian language version of their books any time soon. (Yes, there are lots and lots of people in both Mongolia and Armenia interested in tech books.) This license allows a geek in Mongolia to translate an O’Reilly book into Mongolian and sell it locally, probably the only way these books will get localized into certain languages in the near future.

O’Reilly is noteable because they’ve already stepped up and released many of their titles as “open books”. (They’ve also been tremendously generous to organizations like Geekcorps, donating piles of returned titles for use in developing nations.) It would be very interesting to see O’Reilly or other innovative publishers start using the Developing Nations license for the bulk of their catalog. There’s understandable concern about reimportation of digital texts… but there’s also an amazing social benefit to having this content available in markets where it’s not otherwise profitable to produce it.

CNN moves Switzerland to where the Czech Republic used to be. No word yet on where the Czechs have gone, or the fate of CERN, several UN agencies and innumerable numbered bank accounts. (Insert “cancelled Czech” pun here.)

I spent most of last weekend hanging out at FOO Camp, the remarkable conference Tim O’Reilly hosts for 200+ geeks in the backyard of his corporate headquarters in Sebastapol, CA. Rather than try to transcribe the various sessions (which would likely require another 48 hours), I thought I’d offer the ideas that have stuck with me a few days later:

My new favorite toy, which I discovered at FOO, is del.icio.us, which appears to be the first online bookmark tool I’ll actually use. It’s convenient (runs as a pair of bookmarklets), flexible (able to put a page in an arbitrary number of folders) and socialble (you can see who else has bookmarked a page and then surf their bookmarks.) My bookmarks page is pretty pathetic at present, but I suspect it will become a form of linkblogging for me pretty soon.

Some of the best conversations of the conference concerned data visualization. Tim O’R and Roger Magoulas gave a talk about the data analysis they perform using Nielsen’s Book Scan and their internal sales data. I was less interested in their discoveries (dot.net rising, J2EE falling, the huge markets for books on consumer software, like PowerPoint) than I was in the tools they were using. Most of the research was presented in Treemap form, using a free Java library – I encountered Treemaps for the first time in Martin Wattenberg’s brilliant Map of the Market – I’m hoping to use the library in my GAP research soon. I also enjoyed some of the wacky ways folks are doing data visualization, bending old tools to new and nefarious purposes – is evidently playing with tools that look at data dependencies in software engineering to model the influence of money in US politics – very much looking forward to see what comes out of this. Also very happy to learn about gGobi, which looks like an amazingly powerful open source graphing and data discovery tool. I also had a great conversation with Scott Davis about mapping software, focusing on tools that can distort maps based on one or more factors – I just found a great example of one at the Electoral Vote Predictor earlier today.

My friend Quinn Norton gave a terrific talk on the economics of Kingdom of Loathing, an extremely goofy massively multiplayer online game that appears to break all the rules on how one should run an online game. I found Quinn’s observation that KoL is basically “a blog game” – the homegrown project of one very quirky guy with the input of lots of his users. Is this a new form of personal publishing for the extremely geeky? I suspect it might be – roughly 40% of all geeks I know have either tried to build a MMPOG, or have plans to build one. KoL is a great lesson that it’s not the graphics or the technology – it’s clever, geeky humor and a set of rules flexible enough to allow interesting behaviors to emerge.

On the subject of simple tools and complex behaviors – I spent a good chunk of my time at the conference listening to some of the discussions between the various blogging tool builders attending the conference. While one of my favorite blog tool developers wasn’t at the conference, many of the major developers were, and I had the sense more than once that, if someone wanted to make a huge change in how blogs worked, if one simply convinced everyone in the room, they’d be well on their way. A subtle, but very cool, possible from Kevin Marks of Technorati – we need a slightly more complex semantics of linking. When I add someone to my blogroll, it doesn’t mean that they said something interesting that day – it’s more a representation of our social relationship. Kevin suggests using a tagging standard called XFN, which distinguishes social links from content ones. He’s also looking for ways to add approval and disapproval to links: i.e., when I link to something now, I’m implicitly endorsing it – that’s problematic when you’re linking to something to critique it.

One of the best talks I attended was Brian McConnell’s on “Hacking Your Way Off the Grid”. Brian has spent the past few years tinkering with his San Francisco home, adding photovoltaic panels to cut electricity bills, solar water heat to his hot tub and solar air heat to certain rooms of the house. He’s tracked each step of the process and calculated how many years it will take to amortize each investment. My brain kept flying off in odd directions – could we use lightweight solar air heating units to warm gers in Central Asia? Can obsolete chip fab plants be repurposed to make PV cells? But Brian’s probably on the right track, thinking of simple changes that could make energy efficient technology cheaper and easier to install for US users, simplying the supply chain and applying lots of silicon valley ingenuity to making this technology accessible. His underlying point: what if US geeks spent as much time hacking and improving personal power systems as we do improving microcomputers? His Powerpoint is online here and very much worth reading.

Danny O’Brien’s talks are always worth listening to – I’m still improving my life by implementing suggestions from his “Tech Secrets of Overprolific Alpha Geeks” talk. This time, he focused on the future of fame, specifically the concept of microcelebrity. Danny’s a good example – if you hang around in certain (extremely geeky) scenes, he’s extremely well known. Outside of those circles, you probably haven’t a clue who he is. He suggests that’s the nature of this new medium. In an era of broadcast media, famous people were famous for everyone who watched TV – in an Internet era, it’s possible (and lots more likely) to be famous for a small group of people who care about what you do.

The most compelling idea he shared: We’re not able to scale our imaginations to accept the fact that there’s billions of people in the world, so we scale our world down to the people we know and care about. That’s why some of the best web links you will click on will open an entirely new world, big enough to have its own gossip columns, celebrities, society pages, etc. (For a good example of some of these links, read Biz Stone’s United Federation of Bloggers post, where he’s pointing to strange and wonderful blogs around the world.

But the most interesting idea at the conference was the conference itself. There’s less (apparent) structure for FOO than for any other conference I’ve ever attended. The OReillians designate certain areas of their office buildings as on or off limits, hire chefs to provide a light breakfast and two big meals a day, provide wifi and a wiki and a big board where people can propose sessions they’d like to lead. And that’s about it. And it works better than almost any other conference I’ve ever attended. I don’t know if the model scales, or if it only works with hand-picked attendees, or if there’s lots more structure behind the scenes that I didn’t see… but it’s remarkable and something that I hope others will learn from in putting together conferences, because it’s a great deal of fun. Many, many thanks to Tim O’Reilly and Sara Winge for making it happen.

Passion of the Present is pointing to a new article out in Germany’s Die Welt, which asserts that the government of Sudan, in cooperation with Syria, has used chemical weapons on people in the Darfur region of Sudan. The story asserts that Sudanese and Syrian military officials met to discuss using chemical weapons in southern Sudan against the SPLA. Because peace talks were underway, tests occurred in western Sudan instead, against Darfur rebels. The report cites eyewitness accounts of frozen corpses with chemical burns arriving in Khartoum for examination by Syrian doctors.

AFP, UPI and Deutsche Welle have picked up the story, though all are reporting it as “Die Welt claims”, evidently waiting for further confirmation.

Die Welt’s report appears to rely on eyewitness reports given to ILAF, an Arabic news website based in the UK, and on “the documents of western intelligence services” presented to Die Welt. (translation from Google’s MT engine.) There’s an article from Deutsche Presse-Agentur in Expatica (a Netherlands-based web news site), which quotes an unnamed German intelligence source, who seems to disagree with the Die Welt story: “We find the details very surprising and would have evaluated them differently”.

Alphabetcity has pointed to a number of articles that speculate that Syria has been moving WMDs to Sudan, perhaps to avoid any future UN weapons inspections.

Obviously, this is story – should it be confirmed – has important and complex implications. Sudan apologists, who see global interest in Darfur as an interest in oil, are denouncing the story as a CIA plot. Folks on the right are asserting that this is evidence that Iraq had chemical weapons, which found their way from Baghdad to Damascus to Khartoum. And, should the story be confirmed, people pushing for US, EU or UN intervention in the Darfur conflict, will certainly point to the use of chemical weapons as another reason to intervene.

If anyone has strong enough Arabic to navigate elaph.com and find the August 2nd story referenced by the Die Welt story, I’d be very interested in reading the eyewitness reports that appear to be behind this story.

Tim Cohen, editor at large for Jo’burg’s Business Day, has an excellent op-ed on Solomon Linda and Mbube. Mbube is the song Linda wrote in 1939, which later became “Wimoweh”, when recorded by Pete Seeger, and “The Lion Sleeps Tonight” when lyrics were added by The Tokens. Linda recieved 10 shillings for the song when he wrote it, and a few royalties from Seeger, but generally didn’t benefit from the song’s immense popularity, and died penniless. His family is now attempting to sue the Disney corporation for royalties for use of the song in “The Lion King”. Thus far, Disney has responded by pointing out that they licensed the song from its legitimate owner.

Cohen’s article references an excellent piece by Rian Malan, written for Rolling Stone, which gives a precisely detailed history oif the song’s journey from South Africa to the rest of the world – it’s reproduced online here.

Zimbabwe has sentenced Simon Mann, alleged Equatorial Guinea coup plotter, to seven years in jail for weapons violations. The pilots of the plane that landed in Zimbabwe got 16 months, the passengers, a year. While the BBC reports that these sentences are “stiffer than the men would have expected”, it’s worth remembering that, as the situation unfolded, Mugabe and crew were making noise about finding ways to charge the alleged mercenaries with capital crimes. No word yet on what fate will befall the other 16 alleged mercenaries held in Equatorial Guinea.

In other Zim news, World Press Review has a pair of excellent articles on Mugabe’s systematic war on dissent. Julius Dawu’s “A Death Knell for Zimbabwe’s Press”, which talks about Information Minister Jonathan Moyo’s threats against “mercenary journalists” – that is to say, any journalist who freelances for a non-state controlled paper. Dawu quotes Moyo:

“We will not allow Bush’s boys in our midst … The situation in Zimbabwe today calls for principled actions without fear or favour, and without succumbing to any threats or intimidation. Mercenaries of any kind, whether carrying the sword or the pen, must and will be exposed and they will suffer full consequences of the law,”

Dawu’s second article on WPR is an overview of press arrests in Zimbabwe in late 2003.

Zimbabwe has sentenced Simon Mann, alleged Equatorial Guinea coup plotter, to seven years in jail for weapons violations. The pilots of the plane that landed in Zimbabwe got 16 months, the passengers, a year. While the BBC reports that these sentences are “stiffer than the men would have expected”, it’s worth remembering that, as the situation unfolded, Mugabe and crew were making noise about finding ways to charge the alleged mercenaries with capital crimes. No word yet on what fate will befall the other 16 alleged mercenaries held in Equatorial Guinea.

In other Zim news, World Press Review has a pair of excellent articles on Mugabe’s systematic war on dissent. Julius Dawu’s “A Death Knell for Zimbabwe’s Press”, which talks about Information Minister Jonathan Moyo’s threats against “mercenary journalists” – that is to say, any journalist who freelances for a non-state controlled paper. Dawu quotes Moyo:

“We will not allow Bush’s boys in our midst … The situation in Zimbabwe today calls for principled actions without fear or favour, and without succumbing to any threats or intimidation. Mercenaries of any kind, whether carrying the sword or the pen, must and will be exposed and they will suffer full consequences of the law,”

Dawu’s second article on WPR is an overview of press arrests in Zimbabwe in late 2003.