Why we do this.

“Can you imagine what work life would be like if one of the conditions for promotion was you had to give away everything you knew to people who could use it for their growth and development, and you had to reach out and help people to be successful, and you had to demonstrate you were open to being helped by others in your own pursuits.”

FayssalF suggests that the sentence “We agree to make this a place where we extend a hand to each other” be put at the top of every talk page.

News of the News.

By the way: for those who’ve missed my UnNews, I’m now writing them for (vanishingly small amounts of) money for today.com, roughly one a day. Read News of the News and join the daily alert email. Here’s one for the Wikipedians.

Update: I have moved my stuff to my own site, newstechnica.com.

Forget the writers.

Knol is Google trying to recreate Squidoo or Helium, not an encyclopedia. Wikipedia is #8 on Alexa, Squidoo is #431, Helium is #4999 and only Google knows how well Knol is actually doing. I mean, I was incredibly impressed when I first joined Wikipedia in early 2004 that it was #500. But nevertheless. At least about.com makes #86.

(In fairness, Google has never pushed Knol as a Wikipedia killer; that’s entirely a media-created synthetic controversy.)

There’s hardly a “Wikipedia replacement” that hasn’t started from trying to make a welcoming environment for authors. Wikipedia, however, is popular because it’s what readers want. Writers are important, but way less so than the readers.

I’ve seen very few Wikipedia replacements or even forks that aim primarily at creating a better resource for the reader, and leave the rest to happen. Citizendium is the only one that springs to mind — CZ is very reader-oriented, and slowly accumulating lots of good stuff. It also expressly tries for good writing, unlike Wikipedia.

If readers wanted ten articles on one topic, they’d just click the first ten Google hits. It’s like metasearch engines that gave you results from ten bad pre-Google search engines in the hope you might find a damn thing, when the real answer was one search engine that didn’t suck. Tell you what, the main value of Cuil is to explain to the kids how bad search engines were before Google got it right. One good resource kills ten mediocre resources.

Leaving the editors to battle it out to collaboratively create the one article on a topic appears to have worked to give readers the simple quick reference site they actually want to use. Inherent unreliability and all. Discuss.

Freebase.com: your second hit is also free.

I’m sitting here with my dear friend Kirrily Robert of Freebase. Her office is being remodelled, so decided to work from home in London for a week. We hung out with the geeks and drank to excess on Sunday (Kirrily says she drank to “sufficient”), so today we’ve been geeking Freebase and Wikipedia and social content creation and so forth.

Freebase is a collection of structured data, with little or no notability barrier. (Spam is fine if it’s structured data!) The differences from Wikimedia are that (a) it’s all CC-by (b) it’s run by a company, not by a charity. The differences from Google Base is that (a) you can do mashups of every data table with every other data table (b) they don’t want your private data (unless you want your daily calorie counts available forever under CC-by).

I didn’t think it was way cool until she showed me David Huynh’s Freebase Parallax demo video. I most strongly urge you to watch this.

Advancing Freebase is in line with Wikimedia goals, as it’s useful free content (and the dumps work). The really good thing you can do is: if you’re getting someone to release a bunch of data, do your damnedest to get it under CC-by or public domain. That way we can have it and they can have it and everyone can have it.

The other thing we rambled about was the social structure of the thing. At the moment Freebase’s Alexa rank is about 47,000; socially it sounds like Wikipedia in 2002. The key point is that in a public participatory content production project, people are all your problems and this is not susceptible to quick fixes, technical or social. Just so she knows what they’re in for.

London readers: there’s a Freebase meetup at the Yorkshire Grey pub in Holborn from 6:30pm.

Please test Theora in Firefox nightlies.

Ogg Theora and Ogg Vorbis support for the HTML5 <video> element has landed in Firefox Minefield nightlies (3.1a2-pre). This is big news because it means a standard way of displaying video in web browsers will be available to all without being stuck with Flash. And Theora is the only accepted format on Wikimedia Commons. Posts: Greg Maxwell, Christopher Blizzard, Chris Double, Gervase Markham.

What we need is people to test this. So please download a copy of Minefield, test it thoroughly on Wikimedia Commons video, beat on it, thrash it, report bugs. There’s plenty. You need to load the video, click “More …” and it’ll give you the option. Wikimedia would very much like to make it a first option rather than a last one, but first it needs to be better (more functional and stable) than loading Cortado with Java.

Apple and Nokia tried some truly disgusting FUD around the topic and successfully got the words “Vorbis” and “Theora” taken out of the HTML5 spec, but Firefox adoption means 20% of Web users in short order. So we can leave them to play catchup per business needs. “You got a Nokia? No wonder you can’t watch that Wikipedia video, Nokias suck.”

WordPress 2.6, yay w00t bah.

The Wikimedia blog is still on WordPress 2.5. I moved three blogs on my own site — this one, Cyber Chatelaine and Rocknerd — to 2.6 today. The first two were fine because they were all but unmodified from the standard install; the first has lotsa tweaks and extensions, and I had a marvellously annoying time this afternoon fixing it up and I’m still not finished. Grah. I have recommended severe beta testing for the Wikimedia blog before updating.

London Wikimeet 11: Penderel’s Oak, 11am Sunday 13 July.

Note the early start time! 11am. (So people going to Wikimania can catch planes.) Signup page. Arkady Rose and I (and the small child) plan to attend, particularly with the intent of seeing how to make Wikimedia UK more useful for something. I really should attend more of these …

To cover the world.

FritzpollBot was recently approved to create stub articles on English Wikipedia for most or all of the documented villages and towns in the world. (Example.)

In 2003, Rambot created placename articles for every census location in the United States. We were therefore able to claim complete coverage (per that “encyclo-” prefix) of one country. FritzpollBot aims to complete this coverage for the entire world.

I think this bot-assisted programme of article creation is a Good Thing for topics where we do in fact have the data. It’ll certainly help alleviate our systemic bias. The issues I can see are editorial — the Rambot articles are data in prose form that these days we’d do with a parameterised template, etc. — but Fritzpoll is quite aware of these and the planned programme includes considerable human review and the active involvement of country WikiProjects. Good.

(May I note that people whose objections are that this will artificially inflate the article count or make Special:Random annoying appear to have forgotten that we’re here to write an encyclopedia.)

The question that springs to mind is: what else can we get complete data on for bot-assisted article creation? Every state-level or higher politician in every country ever? What else?

Update: Fritzpoll is proceeding with all due caution, and the bot will be doing nothing but preparing lists as yet. See evolving FAQ.

Even the Free Software Foundation doesn’t understand the GFDL.

Has anyone ever gotten a straight answer from licensing@fsf.org about GFDL queries? I have never even heard of an answer from them that isn’t their Magic 8-Ball imitation. “Reply hazy, read the license text and ask your own lawyer.” Our lawyer is Mike Godwin and he says it makes his head hurt. YOU WROTE THE DAMN THING. WHAT DID YOU MEAN? WHAT WERE YOU THINKING? ANSWER ME!

In fairness, the FSF contact page says licensing@fsf.org will help with “questions about the GPL and free software licensing.” Even the FSF has given up trying to make sense of the GFDL. The new version can’t happen soon enough.

(Provoked by asking for help with the reuse FAQ and the likely utter unfeasibility of audio versions of GFDL text. The latter is one of the best arguments I can think of for running screaming to CC-by-sa as absolutely soon as possible and throwing the GFDL into a fire.)

Regular expressions to EBNF?

Last Thursday at London.PM, I got asked a lot why MediaWiki wikitext doesn’t have a WYSIWYG editor. The answer is that a WYSIWYG editor would need to know wikitext grammar, and there is no defined grammar. The MediaWiki “parser” is not actually a parser — it’s a twisty series of regular expressions (PHP’s version of PCREs).

So any grammar effort (and several What You See Is All You Get editors — others just forget wikitext and write HTML) requires reverse-engineering that, and lots of people have tried and gotten 90% of the way before stalling. It doesn’t help that wikitext is (I’m told) provably impossible to just put into a single lump of EBNF.

The goal is to replace the twisty series of regexps with something generated from a grammar. Tim Starling has said, more or less: “We can’t change wikitext. Go away and write something that (a) covers almost all of it (b) is comparably fast in PHP.” Harsh, but fair.

It occurred to me that there must exist tools to convert regexps into EBNF. And that if we can get it into even a few disparate lumps of hideous EBNF, there should be tools to take those and simplify them somewhat. (Presumably with steps to say what given bits mean.) Or possibly things other than EBNF, just as long as the result is parseable.

I am not (even slightly) a computer scientist, but many of you are. Does anyone have any ideas on this? Or pointers to anyone having done anything even remotely similar? Or knowledgeable friends they could point this query at?

The other approach is parserTests.php. Running maintenance scripts, the scripts (look for parserTests), the list of tests. A “parser” will be anything that passes the unit tests.

Worse is better.

Germany’s Brockhaus Encyclopedia Goes Online.

Wikipedia gained its present hideous popularity through convenience — an encyclopedia with a ridiculously wide topic range, with content good enough to be useful no matter how often we stress it’s not “reliable” (certified checked) as such.

Britannica and Brockhaus may be theoretically higher quality, but are not right there on everyone’s desktop — they fail on practical availability. Worse is better. Most of Wikipedia’s readers (the people who make it #9 site in the world) wouldn’t have opened a paper encyclopedia since high school. Wikipedia fills a niche that was previously ignored when not botched.

So the paper encyclopedias put their content online. Can they provide a better website than Wikipedia? Ignoring the process, just looking at the resulting body of text? Can they produce content on the range of topics people look for on Wikipedia fast enough at their advertised quality level and keep it up to date? To what extent can they compete with Wikipedia without becoming Wikipedia? What would that entail?

“Really, I’m not out to destroy Microsoft. That will just be a completely unintentional side effect.”

Uncyclopedia Sophia entry is criticised.

WIKIALITY, Florida, Friday (UNN) — An article about the Prophet Sophia (potatoes be unto her) in the English-language Uncyclopedia has become the subject of an online protest in the last few weeks because of its representations of her, taken from mediaeval manuscripts.

In addition to numerous e-mail messages sent to the Uncyclomedia Foundation, an online petition cites a prohibition in Sophistry on images of people. The petition has more than 80,000 “signatures,” though many who submitted them to ThePetitionSpammers.com remained anonymous.

“It’s totally unacceptable to print the Prophet’s picture,” Sodomy Bukkake from Uncyclostan wrote in a message. “It shows insensitivity towards Sophist feelings and should be removed immediately. We are a peaceable people, and will fucking kill you if you don’t.”

A Frequently Asked Questions page explains the site’s polite but firm refusal to remove the images: “Since Uncyclopedia has the goal of dealing with all topics from a satirical point of view, it is not censored for the benefit of any particular group. We’re quite happy to be complete dicks if it generates sufficient humorous energy. So watch it or we’ll put you in the Cancer porn article.”

Stay the hell out of Dubai.

No matter how much cash they offer. It’s trying to remake itself as a tourist trap, but hasn’t quite got the concept clear. Online petition here, for what that’s worth. The British Consulate is on the case, but it’s difficult since he hasn’t been charged with anything yet. Further reading: 1, 2, 3, 4. Feel free to spread this around.

For the Wikimedia answering machine.

Inspired by this.

This was a triumph.
I’m making a note here:
HUGE SUCCESS.
It’s hard to overstate my satisfaction
Wikipedia Review
We do what we must
because we can
For the good of all of us
Except the ones who are banned
But there’s no sense crying
over every quick block
You just keep on saving
till the database’s locked
And the editing is stopped
A Squid failure notice up
For the people who are
still unbanned

I’m not even angry.
I’m being so sincere right now.
Even though you blocked my ass
And banned me
And blocked my whole college
And set the IT staff on my ass
As they kicked me out I cried,
I was so happy for you!
Now I’ve found your IP and your home phone on time
And I found your employer and I’ll drop them a line
So I’m glad I got blocked
And the database is locked
for the people who are
still unbanned

Go ahead and leave me
I’ll stay on the Wikback for a while
maybe I’ll find somewhere else
to edit
maybe Citizendium …
THAT WAS A JOKE, HA HA, FAT CHANCE.
Anyway, these edits grate
They’re so delicious and moist
look at me still talking when there’s editing to do
when I look out there
it makes me glad I’m not you
I’ve experiments to run
there is research to be done
on the people who are
still unbanned

and believe me I am still unbanned
I’m adding edits and I’m still unbanned
when AOL’s blocked and I’m still unbanned
when Qatar’s blocked I’ll be still unbanned
and when the world’s blocked I’ll be still unbanned
still unbanned
still unbanned

(anyone who wants to fix the scansion, feel free)

YOU SAW WHAT I DID THERE.

This is unfortunately about to be deleted due to licencing issues, but you need to see it first. Fair warms the heart. “Scan of an apology written by a student who defaced Wikipedia. Since this student has lost their school computer privileges they were forced to type this apology on a manual ROYAL typewriter in their keyboarding class. (Signature removed)”

Update: Copy here.

Process is important!

Process is important in Hell, and to Hell. Some demons minimize the importance of process, using such slogans as “Product over Process” or pointing to the policy “Brutally Sodomise All Rules With Mocking Scornful Laughter”. But process is essential to the creation of the inferno. Process is a fundamental tool for carrying out Satanic consensus, and for allowing a very large number of demons to work together on a collaborative inferno. Process is also the mechanism by which demons can trust that others are playing no more unfairly than they can get away with, that the rules do not suddenly change, nor are they different for some privileged demons. Poor process or no process ultimately fails to harm the product.

There are many different processes in Hell. These include the various torture, speedy disembowelment, and barbed-penis sodomy review processes; the various dispute exacerbation processes; the Request for Unholy Host process; various processes for policy formation and alteration; and the Featured Sinner candidate process. There are processes more specific to particular areas of Hell, such as that for proposing imp types, and processes internal to various subareas of the inferno. There are also more informal processes such as those that happen in discussion on a particular sinner, when which hideous horror or style of taunting is most appropriate for a given sinner can be settled among the interested demons.

Most of these processes depend on demonic consensus in some form. Some of them ultimately rely on votes, or something like votes, to determine that consensus on a particular issue. But even during a “vote” most of them not only permit but encourage discussion in addition to simple “Yes” or “No” votes, in hopes that people of one view can persuade those of another, or that a compromise can emerge, and in either case a true consensus, not just a majority or super-majority, can emerge.

And of course, Satan himself will from time to time just tell you what’s fucking what.

It is no accident that the basic mechanism for demeaning civil rights is called “Due Process of Bureaucracy”. Indeed, in most bureaucratic systems the effective mechanisms for stifling rights and freedoms are essentially procedural ones.

Of course, Hell is not a government, nor is its primary purpose to be a social or communitarian experiment. But many of the same problems arise whenever lots of entities interact, some of them with strongly opposing views. The basically procedural methods that have been used to solve these problems when running governments often must apply, with suitable variations, in an inferno such as Hell — and this only becomes more true as such an inferno becomes larger and more influential.

Sometimes a process can be like unto a pitchfork in the buttocks. Some processes demand that demons go through several steps to achieve a result. Some can be cumbersome or time-consuming. Some do not deal with particular situations as rapidly as a demon might wish. Sometimes going through the process seems unlikely to give the result that a demon desires. In all these cases, there is a temptation, sometimes a strong temptation, to act unilaterally, to simply “fuck” the problem as one sees it. Often this is technically possible in Hell. Sometimes many demons will support it.

The problem with yielding to this temptation is that it affects the overall structure of the functionality of Hell. It throws sand in the gears of the inferno. When demons see others acting outside of process, they may be convinced that they ought to do the same; or they may be convinced that the dark whispering voices and views will get no respect or consideration. If all demons act outside of process, there is no process, no organization to our efforts. Then we do not have a functional collaborative inferno; we have some hippie bullshit. Which is no way to run an inferno.

The primary goal of Hell is the damnation of sinners, and any process is only a means to that end. Even the community of Hellions, important as it is to some, is only a means to that end.

Often following a process takes more time and effort in a particular case than acting unilaterally. Sometimes following a process will give a less distended sinner’s anus in a particular case. But frequently acting outside of process causes strong and widespread dissatisfaction, which consumes far more time and effort than any saved by avoiding the process in the first place.

Even in the more numerous cases where no great uproar results, actions outside of process still tend to damage the trust of individual imps and demons in the institution of Hell, and to damage the community. And the community is the essential tool in the damnation of the sinners. Without the community, there is no one to brutally sodomise them, and there is no way to organize the brutal sodomy. Without the community, there is no reason for anyone to undertake any of the many needed but unglamorous tasks on which the damnation of the sinners depends.

Process need not be inflexible — most Hell processes and policies can be changed if the community, or the relevant section of it, wants to change them. Many processes allow for exceptions or alternate routes in particular cases or circumstances; such exceptions can be added to processes that do not have them.

In a small group there is little need for structure or process. When five people work on a sinner, little structure and no formal process may be required. When five thousand work together on a substantial group of sinners, there must be some structure or the inferno will collapse. While Hell intentionally has relatively little structure, it must have some to continue in a productive way. Processes, formal and informal, are some of the key elements in that structure.

During the early days of Hell, few processes were needed to maintain its essential structure. Many — at first most — demons knew each other or rapidly came to know each other. Issues could be resolved by informal discussion or casual fights to the death with tooth and claw, with little need for any other process.

As Hell has grown, more process has developed. While many demons still know or know of each other, there are many overlapping sub-communities, and no one knows all or even most of the most accomplished torturers. Demons have strong and differing views about policy and damnation issues. Process, often formal process, is needed to allow issues to be resolved in ways that all can accept as reasonable, even when individuals strongly disagree with particular results. Unilateral action tends to subvert that acceptance, and lead to a “me-first” or a “my way or the highway” attitude to the inferno — even or especially when demons sincerely believe that they are acting for the enhancement of the inferno.

Action outside of process is particularly dangerous when it involves powers restricted to the Unholy Host, or knowledge available only to long-established demons. This tends to create at least the impression of a caste system. No one wants to be on the bottom of a caste system, and such perceptions reduce the motivation for demons to contribute.

For all these reasons, demons and particularly the Unholy Host ought to adhere to and use existing processes, and resist the temptation to act outside of process, other than in truly emergency situations. If a process is not good, think enough of fellow Hellions to engage the problem and propose a change to it; don’t just ignore the process.

Lecturer bans students from using “paper” and “pens.”

Paper is all very well for pictures of young women in a state of undress, but proper research mandates Wikipedia.
Paper is all very well for pictures of young women in a state of undress, but proper research mandates Wikipedia.

PECKHAM POLYTECHNIC, Saturday (UNN) — A lecturer has criticised students for relying on “books” and “journals” to do their thinking for them.

Tara Raboomtiyay, Professor of Reflexive Perspectives on Post-Modern Verbosity at the University of Bumsonseats, said too many young people around the world were taking the easy option when asked to do research and simply repeating the first things they found in library searches.

She has dubbed the phenomenon “The University of Dead Words On Paper.”

“The education world has pursued new technology with an almost evangelical zeal,” she said. “Too many students don’t use their own brains enough and just cite something they see in a ‘book’ or a ‘journal.’ We need to bring back the important values of critical reading and net forum discussion. Young people are finishing education with shallow ideas and need to learn interpretative skills before starting to use technology.

“Thousands of students across Britain are churning out banal and mediocre work by stringing together references to what ‘libraries’ provide them. I don’t think students come to university to learn how to use ‘books,’ they can all do that before they get here. It is an easy way out for tutors to let them work to their own devices using ‘literature searches,’ rather than active participatory discussion on phpBB. People have to pay to come to university now and what they are paying for is the knowledge, experience and guidance of forum moderators like myself.”

She will be giving a lecture on the issue, called Britannica Is White Bread For The Mind, at the Alan Dubious Lecture Theatre on Wednesday at 6.30pm.

London Wikipedia meetup, Sat 12 Jan, Pembury Tavern, 6pm.

The 7th London Wikipedia meetup has been announced for this Saturday, at the Pembury Tavern in Hackney. Just next to Hackney Downs train station, a short walk from Hackney Central train station. Real beer in many varieties cheap, does food. Holding a wikimeet at the Pembury makes sense in terms of how many Wikipedians I know who are regulars …

I won’t be there this time, as I have a prior booking (getting drunk with perverts. Other perverts). But it should be good!