Comments on: Help us find some language data http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/ EthanZ's musings on Africa, media and international development Wed, 08 Feb 2012 20:45:31 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: Stian Haklev http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1810523 Stian Haklev Wed, 18 Nov 2009 20:52:17 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1810523 Interesting project. I think browser settings are not very helpful however. A huge percentage of Norwegians just download English Firefox or Internet Explorer, and never bother to set the language setting - many use Windows in English too, but they might still prefer to read in Norwegian, etc. Correct me if I am wrong, but the language setting never seem to have "taken off" - doesn't seem like many pages offer various language alternatives based on this setting... which would be confusing as well, since so few people bother to set it. The Canadian government, for example, insist on always offering me huge buttons for English and French, although this theoretically would be an ideal use for the language setting. (Although I suspect that part of it is also the need for showing off their bilingualism. If English-speakers could just surf seamlessly in English, without ever noticing that there were French versions available, the French-speakers might find that situation less than optimal: even if they had exactly the same experience with French). Interesting project. I think browser settings are not very helpful however. A huge percentage of Norwegians just download English Firefox or Internet Explorer, and never bother to set the language setting – many use Windows in English too, but they might still prefer to read in Norwegian, etc.

Correct me if I am wrong, but the language setting never seem to have “taken off” – doesn’t seem like many pages offer various language alternatives based on this setting… which would be confusing as well, since so few people bother to set it. The Canadian government, for example, insist on always offering me huge buttons for English and French, although this theoretically would be an ideal use for the language setting.

(Although I suspect that part of it is also the need for showing off their bilingualism. If English-speakers could just surf seamlessly in English, without ever noticing that there were French versions available, the French-speakers might find that situation less than optimal: even if they had exactly the same experience with French).

]]> By: Liz http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1810333 Liz Wed, 18 Nov 2009 14:09:02 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1810333 You might get some pointers from Nicholas Ostler (linguist and author of the book Empires of the Word). In his book he makes some attempt to classify world languages by number of native and second language speakers, which I haven't seen in other places. I don't know what he's worked on recently, but he seems to have given this issue some thought anyway. You might get some pointers from Nicholas Ostler (linguist and author of the book Empires of the Word). In his book he makes some attempt to classify world languages by number of native and second language speakers, which I haven’t seen in other places. I don’t know what he’s worked on recently, but he seems to have given this issue some thought anyway.

]]> By: Ethan http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1802613 Ethan Fri, 13 Nov 2009 17:27:54 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1802613 Thanks for the suggestions, folks and for the pushback. I'm aware that it's an oversimplification to build a model in which people speak a single language, and I think Tony's right to point out that people are frequently capable of interacting without achieving fluency in a language. The sort of transactional border-crossing Tony outlines in his comment is exactly what I'd expect to be seeing as made possible by the internet. In rural Montana, it's now possible for the cricket fan to read cricinfo.com, one of the most popular sites in the world. That there's very little readership for cricinfo.com in Montana is an obvious, but also interesting finding - the availability of the information is important for the small fringe of people interested in that content, but most people don't choose to access that information. What's more interesting is that, in some cases, people do choose to access information that's assembled abroad - in UAE, for instance, many of the most popular news sites are not focused on Gulf audiences. This reflects the fact that the residents of UAE are primarily immigrants, focused on news from home. A language prevalence model - like the one we're looking for - won't be accurate. We know that. The hope is to find one that's representative on a gross scale. I.e., it would tell us that Canada has a lot of English speakers, but isn't the "anchor nation" for English, and has a lot of French and Mandarin speakers, but doesn't anchor those languages either. That would then help explain the high degree of non-locality for news. I don't deny the validity of the Beatles-lyric conversation in Prague (actually, you can have quite a rich dialog by just quoting Beatles lyrics...) but think that behavior is going to be a rounding error when considering tens of millions of pageviews in Prague of people reading Czech newspapers... In other words... to get a set that lets us correlate, we're going to have to simplify, and perhaps to oversimplify. That said, the links gathered here already are a great introduction to some of the complexities that orbit this issue. Thanks for the suggestions, folks and for the pushback.

I’m aware that it’s an oversimplification to build a model in which people speak a single language, and I think Tony’s right to point out that people are frequently capable of interacting without achieving fluency in a language.

The sort of transactional border-crossing Tony outlines in his comment is exactly what I’d expect to be seeing as made possible by the
internet. In rural Montana, it’s now possible for the cricket fan to
read cricinfo.com, one of the most popular sites in the world. That
there’s very little readership for cricinfo.com in Montana is an
obvious, but also interesting finding – the availability of the
information is important for the small fringe of people interested in
that content, but most people don’t choose to access that information.
What’s more interesting is that, in some cases, people do choose to
access information that’s assembled abroad – in UAE, for instance,
many of the most popular news sites are not focused on Gulf audiences.
This reflects the fact that the residents of UAE are primarily
immigrants, focused on news from home.

A language prevalence model – like the one we’re looking for – won’t
be accurate. We know that. The hope is to find one that’s
representative on a gross scale. I.e., it would tell us that Canada
has a lot of English speakers, but isn’t the “anchor nation” for
English, and has a lot of French and Mandarin speakers, but doesn’t
anchor those languages either. That would then help explain the high
degree of non-locality for news. I don’t deny the validity of the
Beatles-lyric conversation in Prague (actually, you can have quite a rich dialog by just quoting Beatles lyrics…) but think that behavior is going to be a rounding error when considering tens of millions of pageviews in Prague of people reading Czech newspapers…

In other words… to get a set that lets us correlate, we’re going to have to simplify, and perhaps to oversimplify. That said, the links gathered here already are a great introduction to some of the complexities that orbit this issue.

]]> By: Tony http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1802365 Tony Fri, 13 Nov 2009 14:22:49 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1802365 There's some great stuff here that participants have posted, but I am wondering whether there isn't a glitch in your assumptions -- in my experience language use tends to be transactional. That is, you use what languages you have in a given environment to accomplish what you need to accomplish (to the degree you can). Case in point (maybe stupid) I can remember having "conversations" with people in Prague circa 1969 that consisted entirely of Beatles lyrics (the lingua franca of the 1960s, even behind the Iron Curtain)--this is sort of Solanasaurus' point, I think. Hal's criterion of attention is also transactional -- pay attention FOR WHAT PURPOSE? That redounds also, I think, to the issue of "cosmopolitanism" -- the Internet facilitates access to "otherness" but that isn't a measure that says there was no interest previously. Your own fascination with sumo, for example, might have been latent (say) in 1960, but there would have been no resources for satisfying that interest. I had a schoolmate (in my tiny Montana mining town birthplace) who wanted IN THEORY to be interested in soccer, but at that time there were simply no resources for gaining information -- no magazines, no newspapers, and certainly no TV (we got two channels, one of them poorly). So the desire was there...unfulfilled. Finally, how much language facility constitutes "fluency"? Do you have to speak English like a college prof to be "fluent"? Or is it enough that you can get done what you want to get done? Especially in a confined environment like the internet... I got fascinated at one point with Persian on-line magazines like "Cappucino" (now defunct, I think) even though I have no Persian (pix I could understand). I can kind of follow Beppo Grillo, even though I miss maybe 80% a lot of the time. I was able to get what Yoanni is about (Cuba blogger) even though my Spanish is high school French with bad endings...and so on. The point is not to boast (pointless, since most of my languages make natives weep either with pain or laughter or both) but rather to point out that langauge use is multi-level, with all kinds of degrees of motivation and purposes. My point -- if I have one? -- is that there is a certain reductionism lurking here. People who want to access information will find a way (case in point -- I know a Latvian kid who learned English from the Cartoon Network, so for a time he spoke fluent Batman), whereas people who have closed minds will stay closed, no matter how many languages they formally possess. There’s some great stuff here that participants have posted, but I am wondering whether there isn’t a glitch in your assumptions — in my experience language use tends to be transactional. That is, you use what languages you have in a given environment to accomplish what you need to accomplish (to the degree you can). Case in point (maybe stupid) I can remember having “conversations” with people in Prague circa 1969 that consisted entirely of Beatles lyrics (the lingua franca of the 1960s, even behind the Iron Curtain)–this is sort of Solanasaurus’ point, I think. Hal’s criterion of attention is also transactional — pay attention FOR WHAT PURPOSE? That redounds also, I think, to the issue of “cosmopolitanism” — the Internet facilitates access to “otherness” but that isn’t a measure that says there was no interest previously. Your own fascination with sumo, for example, might have been latent (say) in 1960, but there would have been no resources for satisfying that interest. I had a schoolmate (in my tiny Montana mining town birthplace) who wanted IN THEORY to be interested in soccer, but at that time there were simply no resources for gaining information — no magazines, no newspapers, and certainly no TV (we got two channels, one of them poorly). So the desire was there…unfulfilled. Finally, how much language facility constitutes “fluency”? Do you have to speak English like a college prof to be “fluent”? Or is it enough that you can get done what you want to get done? Especially in a confined environment like the internet… I got fascinated at one point with Persian on-line magazines like “Cappucino” (now defunct, I think) even though I have no Persian (pix I could understand). I can kind of follow Beppo Grillo, even though I miss maybe 80% a lot of the time. I was able to get what Yoanni is about (Cuba blogger) even though my Spanish is high school French with bad endings…and so on. The point is not to boast (pointless, since most of my languages make natives weep either with pain or laughter or both) but rather to point out that langauge use is multi-level, with all kinds of degrees of motivation and purposes. My point — if I have one? — is that there is a certain reductionism lurking here. People who want to access information will find a way (case in point — I know a Latvian kid who learned English from the Cartoon Network, so for a time he spoke fluent Batman), whereas people who have closed minds will stay closed, no matter how many languages they formally possess.

]]> By: Jonathan O'Donnell http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1801579 Jonathan O'Donnell Thu, 12 Nov 2009 23:52:15 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1801579 I think that you will find some interesting ideas in the paper below. They looked at using other census data to predict the different languages spoken in Australia. Demographics being, after all, the science of prediction based on fuzzy, inaccurate, incomplete data that is changing over time.

Modelling languages other than English spoken in Australia using census data, Lujuan Chen, Paul Romanis and Katie Palin, Research paper 1351.0.55.002, Australian Bureau of Statistics, Presented at the Australian Population Association 12th Biennial Conference, 15–17 September 2004, Canberra, Australia [PDF, 241 kb]

http://www.abs.gov.au/AUSSTATS/free.nsf/log?openagent&1351055002_2000-01.pdf&1351.0.55.002&Publication&3E2A4663F70CDF38CA256FD4007F2A0B&&2000-01&31.03.2005&Latest

The Australian Bureau of Statistics also has Standards for Statistics on Cultural and Language Diversity, but they don’t seem to actually talk much about language, but does talk about ethnic diversity, which is part of what you are talking about when you talk about cosmopolitanism, I think.

http://www.abs.gov.au/AUSSTATS/abs@.nsf/Latestproducts/1289.0Contents20001999?opendocument&tabname=Summary&prodno=1289.0&issue=1999

Perhaps you would be better off with the Australian Standard Classification of Languages (ASCL), 2005-06 (go to the Downloads tab for the good stuff).

http://www.abs.gov.au/AUSSTATS/abs@.nsf/allprimarymainfeatures/4293E19B52AC5377CA25703E00045C2E?opendocument

Some of the complexity in this issue is expressed by this submission to an Australian Bureau of Statistics review by the Federation of Ethnic Community Councils of Australia in 2004.

http://www.fecca.org.au/submissions/2004/submissions_2004004.pdf

Keeping all that in mind, here is a lovely spreadsheet of languages spoken in Australia over time (1996, 2001 and 2006 censuses).

http://www.censusdata.abs.gov.au/ABSNavigation/download?format=xls&collection=Census&period=2006&productlabel=Language%20Spoken%20at%20Home%20by%20Sex%20-%20Time%20Series%20Statistics%20(1996,%202001,%202006%20Census%20Years)&producttype=Census%20Tables&method=Place%20of%20Usual%20Residence&areacode=0

Have you thought about cross-checking using other data sets? One of my colleagues here at RMIT is looking at remittances, which is a great way to look at links between countries at the family level. Match it up with trade data and you get a different view of cosmopolitanism. But maybe I don’t understand what you are looking for.

PS: Sorry about the crappy long links, but I didn’t know if I could embed HTML tags into this comment box and didn’t want to have to do it twice.

Jonathan O’Donnell
Nautilus Institute
RMIT University
jonathan.odonnell@rmit.edu.au

]]> By: Kara Pecknold http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1801532 Kara Pecknold Thu, 12 Nov 2009 23:07:55 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1801532 I came across this some time ago and I'm not sure if it is of any help or use to you. But for its worth (as one very interested in your research), I pass it along: http://portal.unesco.org/culture/en/ev.php-URL_ID=35344&URL_DO=DO_TOPIC&URL_SECTION=201.html I came across this some time ago and I’m not sure if it is of any help or use to you. But for its worth (as one very interested in your research), I pass it along:
http://portal.unesco.org/culture/en/ev.php-URL_ID=35344&URL_DO=DO_TOPIC&URL_SECTION=201.html

]]> By: Lester Bolicenni http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1801523 Lester Bolicenni Thu, 12 Nov 2009 22:54:50 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1801523 Sure, you already had a look on that but i thought it was right to mention: as EU emphasizes multilingualism in Europe, and since a specific commission has been created for that purpose, you would find interesting stuff on their website. Surveys about the topic you ask for: http://ec.europa.eu/education/languages/useful-links/ul3323_en.htm#Studies As well, here you can find a map of european minorities: http://www.eurominority.org/version/maps/map-langues.asp I can imagine several regional organisations should provide such resources. Lester Sure, you already had a look on that but i thought it was right to mention: as EU emphasizes multilingualism in Europe, and since a specific commission has been created for that purpose, you would find interesting stuff on their website.

Surveys about the topic you ask for:

http://ec.europa.eu/education/languages/useful-links/ul3323_en.htm#Studies

As well, here you can find a map of european minorities:

http://www.eurominority.org/version/maps/map-langues.asp

I can imagine several regional organisations should provide such resources.

Lester

]]> By: Solanasaurus http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1801487 Solanasaurus Thu, 12 Nov 2009 22:17:29 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1801487 One thing which is fun to look at is what languages people have their computers/browsers set to when they visit Global Voices in English (according to Google Analytics). Current top 10 en-us en-gb fr en zh-cn de es ru zh-tw ar-sa When you compare it to our current top 10 visitor countries you can see that the languages don't always match the country ranking, which makes you wonder (generally) what bits of the data are inaccurate, or whether it says something about language behavior of people who visit Global Voices wherever they live - and of course what language services are comfortably available in various computer operating systems. United States United Kingdom Canada India China Australia France Germany Philippines Japan Anyway, that's just GV. But I thought if you were able to get lots more data on global internet traffic as well as language settings you might be able to deduct something - about some things. One thing which is fun to look at is what languages people have their computers/browsers set to when they visit Global Voices in English (according to Google Analytics).

Current top 10

en-us
en-gb
fr
en
zh-cn
de
es
ru
zh-tw
ar-sa

When you compare it to our current top 10 visitor countries you can see that the languages don’t always match the country ranking, which makes you wonder (generally) what bits of the data are inaccurate, or whether it says something about language behavior of people who visit Global Voices wherever they live – and of course what language services are comfortably available in various computer operating systems.

United States
United Kingdom
Canada
India
China
Australia
France
Germany
Philippines
Japan

Anyway, that’s just GV. But I thought if you were able to get lots more data on global internet traffic as well as language settings you might be able to deduct something – about some things.

]]> By: David Sasaki http://www.ethanzuckerman.com/blog/2009/11/12/help-us-find-some-language-data/comment-page-1/#comment-1801468 David Sasaki Thu, 12 Nov 2009 21:59:19 +0000 http://www.ethanzuckerman.com/blog/?p=3386#comment-1801468 Having just finished a conference where every participant spoke at least three different languages fluently (and each about a third of the time it seemed), language data is going to keep getting fuzzier and fuzzier. From what I've seen ethnologue is always cited as the authority, but Wikipedia has a list of other sources: http://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers#References I'm not sure if there is a global aggregate of national census data with regard to language, but that seems like it would be the most accurate. Two footnotes: 1) There is a difference between the news we consume and where we consume it. Measuring site visits alone confuses that. 2) I think multilingualism is only one aspect - and probably not the most important aspect - of cosmopolitanism. After all, you're one of the most cosmopolitan guys I know ... but also the only multilingual cosmopolitan I know. Have you seen http://peace.facebook.com? Now there's a fascinating data set! Having just finished a conference where every participant spoke at least three different languages fluently (and each about a third of the time it seemed), language data is going to keep getting fuzzier and fuzzier. From what I’ve seen ethnologue is always cited as the authority, but Wikipedia has a list of other sources:

http://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers#References

I’m not sure if there is a global aggregate of national census data with regard to language, but that seems like it would be the most accurate.

Two footnotes:

1) There is a difference between the news we consume and where we consume it. Measuring site visits alone confuses that.

2) I think multilingualism is only one aspect – and probably not the most important aspect – of cosmopolitanism. After all, you’re one of the most cosmopolitan guys I know … but also the only multilingual cosmopolitan I know.

Have you seen http://peace.facebook.com? Now there’s a fascinating data set!

]]>