“Look, we’re all impressed with Wikipedia. But you seem to be saying that to edit I have to put your project ahead of my day job; and so I think you guys are just a bit crazed.”

“Right both times.”

“And you’re now telling me I have to flack for the opponents of the guy I am paid by, and put their criticisms into due form in the the way that, frankly, they are too dumb to do, using the skills I have but against the brief I have been given.”

“Yup, that’s what it says on the page about neutrality.”

“Well … where I come from … words fail me …”

This is really not the beginning of a beautiful friendship.

PR people have a legitimate issue: Wikipedia is this huge scary thing people don’t know how to approach, and sometimes our articles are in fact rubbish. How do you fix that? I get asked this a lot and have a ten-minute reply that I play back like a voice macro, on the themes of:

• be totally upfront about who you are, where you’re from and who you’re working for;
• only ever edit the talk page, never the article itself. Not ever;
• imagine every little thing you do on Wikipedia being picked over by the newspapers a year from now, with your name and photo on it — the papers love this stuff;
• if there’s a serious problem with an article, like defamation, email info@wikimedia.org and someone experienced will look at your issue and take it seriously. (I can’t guarantee any particular response, but I can guarantee it will be taken seriously.)

I visited the Wikimedia UK office on Tuesday and chatted with Stevie Benton (the new media person), Richard and Daria about this topic. In all our experience, even sincere PR people seem biologically incapable of understanding “conflict of interest,” but will understand generating bad PR.

The approach we could think of that could work is: “if you’re caught in what other people think is a conflict of interest, your name and your client’s name are mud.” Any PR who doesn’t comprehend that deserves what happens to them.

Think it’ll help? I realise that you can’t legislate stupidity or malice out of existence … it also looks too much like a threat, which isn’t the intention. Better ideas with some chance of working are much needed. Wikipedians? PR people?

Update to clarify: my comments are strictly advisory and based on watching the press absolutely crucify PR people who have edited clients’ articles, which becomes bad PR for the client — even if what they did was within Wikipedia rules and they arguably didn’t deserve it. I’ve been repeatedly amazed at just how upset the press and the public (e.g., people I talk to) get about this, much more than the actual Wikipedians do.

I expect discussions will involve a bit of the good people on both sides apologising for the actions of the not so good ones …

I spent two days doing radio. The analogy I bludgeoned into the ground (nicked from Cory Doctorow): “Banks notice that bank robbers use cars for getaways. Their solution: we have to ban cars. You say that’s ridiculous and damaging and won’t work anyway. They say ‘YOU JUST LIKE BANK ROBBERS!’” I’m told I suitably scared people on Radio 4 PM (38:50 on) with the consequences for UK business, though I could have done better. (Must not engage with the lying RIAA bullshit. Must not engage with the lying RIAA bullshit. Just repeat the soundbites. Must not engage with the lying RIAA bullshit.)

(If you’re an experienced UK Wikipedian who thinks they could acquit themselves reasonably well and have no problem with being a very minor public figure, please make yourself known on the wikimediauk-l mailing list and/or local meetups and you may get lined up for next time stuff explodes, the press want a random Wikipedian, etc. And feel free to casually talk to your local media about Wikipedia as the occasion arises.)

I’m amazed that several hundred Wikipedians largely agreed on anything at all. The Wikipedia community will fiddle around the edges of an issue forever and then not do anything. If you say to a bunch of Wikipedians “the sky is blue”, they’ll come back with a hundred pages of referenced counterexamples. The key skill to being a Wikipedia editor appears to be generalised cross-domain bikeshedding.

I hope we don’t do this, or anything like it, again any time soon. Our power requires not being used very often at all. But then: if Wikipedia says you suck … you really, really suck.

$200,000 to $300,000 a year is, at the very least, the same as hiring a very good engineer for a year (and paying all the taxes and benefits and costs and still having a lot of money left). It was enough money to finally push us into doing our own maps.

(More detail. They didn’t actually spend less money, but they got much more control and a much better result for the same money.)

I’ve long thought OpenStreetMap would have fit Wikimedia’s portfolio wonderfully — it’s a marvellous example of a project doing really well with the Wikipedia model, without being a linked entity. (And thus helping our mission without us doing the work.)

I remember a London Dorkbot presentation in 2004 on OSM — a friend who was working for Multimap pooh-poohed the idea that OSM could ever achieve a usable-quality map. I had been involved in Wikipedia for a few months at that time and considered this immediately obviously wrong, having seen what a few people just chucking in what they knew could achieve even at that stage.

OSM now has a foundation. Seeing as their server appears to be melting, they could probably do with a quid or two. They’re not yet officially a UK charity, but WMUK achieving charitable status does makes this more feasible.

There should be no chance to gouge for this sort of content. What other rent-seeking business models can the Wikipedia model destroy? Update: List of things that need to be free.

(And don’t forget the official Planet Wikimedia aggregators and the Open Wiki Blog Planet … though that appears to have stalled in July.)

The question is: what other utterly and unambiguously public domain works is JSTOR charging outrageous sums for? And can we get copies to free?

(JSTOR are not a soft and fluffy nonprofit. They are an exceedingly murky nonprofit.)

Some are already uploaded to Commons. Here is the Wikisource project.

(I’m in the UK, where the state of the law concerning slavish copies of public domain works is still in a state of quantum uncertainty, and no-one quite wants to joggle it until the lay of the land is clearer. If I were in the US, where the law is unambiguous and tested, I’d be uploading as we speak.)

Casual meet with some Wikimedia UK people last night. As is standard whenever two or three Wikimedians meet, we conspired to fix everything that’s wrong with Wikipedia and Wikimedia! Perhaps 1% will ever be attempted. One project that desperately needs participants is Wikisource, particularly so that it can handle incoming documents from museums and archives — Commons is doing okay with the images.

Last week I took The Wolfgang Press from two paragraphs to a proper article. Yesterday it was on the front page Did you know? section for six hours and got 3065 hits, compared to its usual 25-50. Not bad for an article about an obscure band approximately no-one cares about. (“Kansas”, from Bird Wood Cage, is a lost goth rock classic. DJs, please play. Thank you.) I’d forgotten how much quiet nerdy fun it is writing and researching a Wikipedia article. Writing about anything artistic on Wikipedia is arse, though, unless you can find critics to quote. Printed ones by preference. Wikipedia’s epistemology is severely broken at the edges, and knowing how it got that way doesn’t actually help.

tl;dr: Reusing GFDL content is unworkably unclear. When you ask the FSF about the tricky bits, they say “better get a lawyer, son.” This egregiously misses the entire damn point. Freedom only for those with copyright lawyers on tap is not freedom.

The GFDL is possibly one of the worst free content licenses ever. The only reason it has not been justifiably buried at the bottom of a swamp with “MISERABLE FAILURE” burnt into its forehead with a soldering iron is because Wikipedia used to use it. Wikipedia used it because Nupedia used it. Nupedia used it only because the CC licenses hadn’t been invented yet.

Literally no one understands how to reuse GFDL content safely, including the FSF. I sent a query about how it applies to aggregates; three months later, the FSF cut’n'pasted their “we have no idea either” response, suggesting you read the license text and consult your attorney. Given Wikimedia’s attorney at the time was Mike Godwin and it made his head hurt too, this strongly suggests no-one left at the FSF wants to think about this thing. (They finally clarified this particular issue in GFDL 1.3.)

In the context of mirroring a widely-edited wiki or a page thereof, its terms are difficult to follow, legally unclear and may be technically impossible to comply with in a comparable degree of safety to the GPL or CC by-sa. (Every copy must have the full 23 kilobytes of licence text attached, about seven pages of single-spaced 12 point. This is not so good for single articles or photographs, and Internet video is likely impossible to reproduce under the GFDL in legal safety. This is, of course, the easiest term of the GFDL to obey. CC by-sa allows the license to be named by reference.) And don’t even ask about images.

Before Wikimedia went Creative Commons, tedious nerds would frequently claim that any given reuser of Wikipedia content was technically violating the GFDL no matter what shrubberies they obtained (thus putting off quite a lot of reusers). And no-one, particularly not the FSF, could provide meaningful advice that could actually be relied upon, because the licence is deeply and fundamentally not fit for purpose.

The GFDL is so unremittingly awful, hard to trust legally and onerous in practice that the Debian project went so far as to throw its hands in the air and declare that it failed the Debian Free Software Guidelines. (The DFSG threat model is “What if an insane or malicious copyright holder comes after a reuser?” This is the question the FSF answers “Better get a lawyer, son.” THIS IS NOT HOW TO BE REUSABLE. The FSF does not answer this concerning the GPL, for instance.) So if you want your open source package in Debian, you probably don’t want the documentation under GFDL.

Use CC by-sa, CC-by or Public Domain, like approximately everyone else in the whole goddamn free content world does. If it’s a software manual, licence or dual-licence it under the same licence as the software itself. You know how some people use shitty handrolled software licenses that are technically free or open but aren’t actually compatible with anything else, and everyone else thinks they’re dicks for it and avoids using their stuff? That’s why you shouldn’t use the GFDL either.

So, in essence, many Wikipedia articles are another way that the work of news publications is quickly condensed and reused without compensation.

I’m sure he’s most pleased that sort of thing will stop and the New York Times will be cited about as often as The Times is these days.

Every journalist I’ve spoken to since 2006 uses Wikipedia as their handy universal backgrounder. Funnily enough, there’s a distinct lack of donations to the Wikimedia Foundation from newspapers and media organisations. How much did the New York Times donate in the fundraiser?

We do this stuff for everyone to use and reuse. Journalists taking full advantage of this is absolutely fine. But claiming we should then pay the papers for the privilege is just a little odious.

I wish the New York Times the full fruits of this bold move.

My experience of volunteer motivation:

1. Running an indie rock fanzine in the 1980s and 1990s. Your writers are writing for free; how do you keep them pumping out good stuff? How do you gently let down the really shit ones?

2. Employed as paperwork administrator for a private school fundraising drive. That was easy because the volunteers had a very specific task: get the people on the donation level below them to donate, knowing the volunteer asking them had donated more than they were asking for. (Work that Pareto principle! This method is so effective it’s pretty much standard.)
3. Editor and then supervisor on a student newspaper. Student papers pretty much exist to be written rather than read; the idea is to get fresh new university students doing something that they can look at and say “I made this!” when they’ve possibly never done anything, ever before. You need a certain amount of discipline to the task.
4. Rocknerd, which is pretty much an indie rock fanzine again, but on the web. In this case, you give opinionated people somewhere to write stuff. I’ve almost no-one else bothering, because everyone’s got a blog of their own.
5. Wikipedia. This is very different because your volunteers are taking up internet space rather than physical space. Also, on Wikipedia the volunteers are … sometimes a bit socially odd. The idea of organising all the facts in the world attracts people from further up the autistic spectrum than you would normally encounter, so some people’s social skills can be brittle. The authority thing is strictly meritocratic — being an admin (which really means “janitor”) or even an arbitrator (which really means “bouncer”) doesn’t cut it; you have to show consistent cluefulness to convince; there is no actual hierarchy. So to get stuff done you have to get very good at convincing people to work on what you want them to.

I’m also slightly amazed at projects like the Puffing Billy tourist railway in Melbourne. Even all the shitwork is done by volunteers, the sort of shitty jobs that turn a paid employee doing them to thoughts of socialist revolution — someone loves the railway so much they get up at three every morning to start the boiler heating! A lot of the tour guides are young folk doing it as work experience (tourism degrees and so forth) or retirees who like to do something for the general good that gets them out and meeting people.

Volunteers will work ten times as hard as any employee, but only because they want to be there and only on things they want to. But that motivation is so fragile, and volunteer effort is not fungible.

The researcher called a few UK people for the Wikipedian viewpoint. As it turns out, we have one — more than a few Wikipedians have bought these things, thinking they were hitherto-unknown new printed sources to use, only to discover their own words on the topic! At prices like $50 for a 10,000 word pamphlet, this is a most unpleasant surprise. It’s also caught a number of slightly famous people who were surprised to find someone had “written” a book about them.

The casual reader encountering these things may not be aware of the business model. These are print-on-demand books, compiled by computer from a list of keywords. No copies exist until someone orders one, at which point a single copy is printed and sent. People aren’t generally aware that POD is very good quality these days — you can send a PDF to a machine and have it spit out an absolutely beautiful perfect-bound book for you, of a standard which previously would have been quite pricey. So Books LLC and Alphascript and whoever manage to eke out a tiny profit on single copies, having worked out a way to spam Amazon.

The books are entirely legal — you can use our stuff without permission, even commercially, and “Please, use our stuff!” is why quite a lot of us do this at all. And we have a link on each wiki to make your own PDF book, and various projects have partnerships with printers like PediaPress. So the main issue is that it’s not being made clear that these books are just Wikipedia reprints.

I tried to stay strictly descriptive of consensus, but I think I could clearly say that we would very much like the publishers and Amazon to make it clearer just what these things are. Thanks.

The problem is there’s no direct action we can really take without hampering the good reasons for reuse of our material. Or scaring people off entirely — it’s hard enough getting across the idea of freely reusable content as it is. We can use publicity about this to spread awareness that we’re all about reusing our stuff, as we introduce civilisation to the notion of reusability as being the normal order of things.

thewub points out on foundation-l:

From March 1st it might be worth contacting the UK Advertising Standards Authority, as their remit is being extended then: http://asa.org.uk/Regulation-Explained/Online-remit.aspx Amazon product descriptions almost certainly fall under “non-paid-for space online under [the marketer's] control”. So a misleading description ought to lead to action. But the issue here is the misleading *lack* of any description. It could be an interesting conundrum for the ASA!

The problem will largely solve itself: if physical copies of Wikipedia articles ever gained any actual popularity, competition would kick in very fast. Even competition on quality would fail, as people sought to design beautiful editions just because they could.

But they won’t. Is the essence of a book the informational content? Is the essence of a book a lump of tree pulp? Is the essence of a book the ideal synergy of the two, creating an object of beauty, wonder and love? The answer people seem to be picking is the first. The Kindle may be a hideously locked-down proprietary money trap, but it’s really quite lovely as a book reader. I read books as PDFs on my netbook, hardly ever picking up my paper copies. A printed general encyclopedia is now a ludicrous idea. “It’s from a printed book!” will soon be as relevant a criterion to sourcing as “It’s on a website!”

Anything made of atoms is a white elephant. I have a four hundred kilogram vinyl record albatross. I will never rip these things, having had a turntable four years and ripped none. Music is digital. Books are digital. Stuff is a curse.

(In that essay, Paul Graham explicitly excludes books from being counted as mere “stuff.” He is wrong.)

Update: Books, LLC responds. They claim Amazon removed Wikipedia links from the listings.

There’s basically no way at this stage for someone to be a better Wikipedia than Wikipedia. Anyone else wanting to do a wiki of educational information has to either (a) vary from Wikipedia in coverage (e.g., be strongly specialised — a good Wikia does this superlatively) (b) vary from Wikipedia in rules (e.g., not neutral, or allow original research) and/or (c) have a small bunch of people who want to do a general neutral encyclopedia that isn’t Wikipedia and who will happily persist because they want to (e.g., Knowino, Citizendium, WikInfo).

Competition would be good, and monopoly as the encyclopedia is not intrinsically a good thing. Indeed, it’s actually quite a bad thing. It’s mostly a headache for us. Wikipedia wasn’t started with the aim of running a hugely popular website, whose popularity has gone beyond merely “famous”, beyond merely “mainstream”, to being part of the assumed background. We’re an institution now — part of the scenery. This has made every day for the last eight years a very special “wtf” moment technically. It means we can’t run an encyclopedia out of Jimbo’s spare change any more and need to run fundraisers, to remind the world that this institution is actually a rather small-to-medium-sized charity.

(I think reaching this state was predictable. I said in 2005 that in ten years, the only encyclopedia would be Wikipedia or something directly derived from Wikipedia. I think this is the case, and I don’t think it’s necessarily a good thing.)

The next question is what to do about this. Deliberately crippling Wikipedia would be silly, of course. The only way Wikipedia will get itself any sort of viable competitor is by allowing itself to be blindsided. Fortunately, a proper blindsiding requires something that addresses structural defects of Wikipedia in such a way that others can use them.

(One idea that was mooted on the Citizendium forums: a general, neutral encyclopedia that is heavy on the data, using Semantic MediaWiki or similar. Some of the dreams of Wikidata would cover this — “infoboxes on steroids” at a minimum. Have we made any progress on a coherent wishlist for Wikidata?)

But encouraging the propagation of proper free content licences — which is somewhat more restrictive than what our most excellent friends at Creative Commons do, though they’re an ideal organisation to work with on it — directly helps our mission, for example. The big win would be to make proper free content licenses — preferably public domain, CC-by or CC-by-sa, as they’re the most common — the normal way to distribute educational and academic materials. Because that would fulfill the Foundation mission statement:

“Imagine a world in which every single human being can freely share in the sum of all knowledge. That’s our commitment.”

— without us having to do every bit of it. And really, that mission statement cannot be attained unless we make free content normal and expected, and everyone else joins in.

We need to encourage everyone else to take on the goal of our mission with their own educational, scientific and academic materials. We can’t change the world all on our own.

So. How would you compete with Wikipedia? Answers should account for the failings of previous attempts. Proposals involving new technical functionality should include a link to the code, not a suggestion that someone else should write it.

I wonder what gay and lesbian employees of Google think of this. I haven’t heard one breathe a peep over the fact that any TVTropes page with the slightest gayness is behind a filter. Censorship, it’s insidious.

But! That’s okay! You’ve all got mortgages.

TVTROPES IS THE UNIVERSAL COUNTEREXAMPLE. YOU CANNOT ADVOCATE ADS ON WIKIPEDIA WITHOUT A KILLER CASE FOR WHY THIS WOULD NOT HAPPEN TO US.

Wikipedia having ads would be the worst possible move for the mission: “Imagine a world in which every single human being can freely share in the sum of all knowledge. That’s our commitment.”

No advertiser in existence would stand for that in practice.

This is a much bigger potential win than many people think. From mediawiki-l in May, a Canadian government official posted how adding a (locally patched) instance of FCKeditor to their intranet got eight times the participation:

In one government department where MediaWiki was installed we saw the active user base spike from about 1000 users to about 8000 users within a month of having enabled FCKeditor. FCKeditor definitely has it’s warts, but it very closely matches the experience non-technical people have gotten used to while using Word or WordPerfect. Leveraging skills people already have cuts down on training costs and allows them to be productive almost immediately.

The geeks refused to believe that not requiring people to wade through computer guacamole worked and that everyone new must be idiots. The poster disabused them of this conceit:

Since a plethora of intelligent people with no desire to learn WikiCode can now add content, the quality of posts has been in line with the adoption of wiki use by these people. Thus one would say it has gone up.

In the beginning there were some hard core users that learned WikiCode, for the most part they have indicated that when the WYSIWYG fails, they are able to switch to WikiCode mode to address the problem. This usually occurs with complex table nesting which is something that few of the users do anyways. Most document layouts are kept simple.

Eight times the number of smart and knowledgeable people who just happen to be bad with computers suddenly being able to even fix typos on material they care about. Would that be good or bad for the encyclopedia?

Now, WYSIWYG has been on the wishlist approximately forever. Developer brilliance applied to the problem has dashed hopes on the rocks every single time. Brilliance is not enough: we’re going to need to apply money.

• We need good WYSIWYG. The government example suggests that a simple word-processor-like interface would be enough to give tremendous results. So that’s an achievable target.
• It’s going to cost money in programming the WYSIWYG.
• It’s going to cost money in rationalising existing wikitext so that the most unfeasible formations can be shunted off to legacy for chewing on.
• It’s going to cost money in usability testing. Engineers and developers are perpetually shocked at what ordinary people make of their creations.
• It’s going to cost money for all sorts of things I haven’t even thought of yet.

This is a problem that would pay off hugely to solve, and that will take actual money thrown at it.

How would you attack this problem, given actual resources for grunt work? What else could do with money spent on it?

Magnus Manske, in his usual manner, has coded up a quick editor whose name I’ve stolen for this post. It’s rough, but it’s a nice working example of some of the way there. WYSIFTW. Screenshot.

Angie Byron did a presentation on getting women into open source which David Eaves spoke further on.

Summary: With open source software, there are people who think “that’s dumb,” there are people who think “I want to see it fixed” and there are people who think “I can do something about it.” The people at the intersection of all three power open source.

On Wikipedia, the intersection point – marked “These people power open source” – is pretty much anyone who knows something and can write it in a coherent sentence. But social structures have evolved to keep it from turning into complete rubbish. Sometimes way too much structure. This means there are all manner of social mechanisms to repel clueless n00bs, since there aren’t the technical or thinking-style barriers there are to coding. And many of these are (IMO) inappropriately strong.

How not to bite the n00bs is a perennial topic on Wikipedia, and I’m currently trying to get some old hands to edit as IPs so they can see how n00b-biting we actually are in practice. Results are disheartening so far.

Some open source projects do similar things to deal with the “Linus doesn’t scale” problem when they attract large volumes of people who can at least write code to a minimal degree. A high quality requirement on code is ony the start — there’s getting attention to your code, jumping through the right hoops, dealing with obnoxious-nerd-stereotype personalities (there’s a reason it’s a stereotype), etc.

But then the problem is what to do when you have more of them than you can deal with — the “Linus doesn’t scale” problem. It’d be nice if the only thing keeping them out was code quality, but it’s silly to claim that’s all there is.

Wikipedia has much the same problem — that intersection, which is tiny for software, is pretty much everyone who knows something and can type for a wiki. So hideously complex social structures have evolved to deal with the distributed crapflood. Many of which are way less than ideal and n00b-bite way too much.

That is: our problem is not getting people into that intersection point — it’s what happens to people in that intersection point, how to keep them from flooding you and how to make sure those mechanisms aren’t in fact damaging your project.

(It’s amazing how much time community nuts’n'bolts uses and how little sense one can have of things actually pushing forward. Wikipedia is the size of a small city. You know how hard it is herding five volunteers? Try getting ten thousand to do any particular thing.)

The last big example I can think of was XFree86 versus Xorg. XFree86 was all but stalled, with Linux vendors having to maintain huge patches themselves because the main project was so slow to accept changes. When Keith Packard, who’s personally driven X for twenty years, finally said “enough” and started organising Xorg, they expelled him.

Compare with Wayland, the new display manager to replace X in Ubuntu and Fedora. (Not a code fork, but in practical terms a developer effort fork.) The three people pushing for Wayland to replace X in Fedora are all Xorg lead developers. “Traitors”? No, people who have the actual aim in mind: making good open source display software.

Xorg remains alive and well even as several lead devs work on its replacement. XFree86 appears to have been abandoned, with no release in two years and no commits since February 2009.

Wikipedia has had any number of forks. Fred Bauder has been with Wikimedia since it was wikipedia.com — his Wikinfo fork has not led to him being regarded as a “traitor” in any way, he’s as highly respected as ever. Wikipedians have always had great interest in its forks and wished them well, including Citizendium. The community regards the forks as family, not enemies. We’re all on the same side: free educational content.

And not to mention that the project crying “treason!” at the word “fork” … started as a fork.

You can’t keep your project together with paranoia. There is no Iron Curtain around an open source project.

(There’s much wackiness around Citizendium at present. I haven’t edited there in three years, but Matt Innis has taken care to block me anyway for writing about them on RationalWiki. Gosh, that’ll sure show me! The point being, of course, internal signalling rather than anything that would affect me at all. “I will not answer any more questions and will ask the Constabulary to delete all discussions that in my view require open debate which is being suppressed here.” You can get running updates and discussion at RationalWiki. Bring your own popcorn.)

Then I got to the end and my jaw dropped when I saw what the author does for a living. Try to read the article without skipping to the end, scrolling down carefully without jumping ahead, it’s worth it.

So. What do we do to distinguish experts from non-experts when we no longer even have credentials as a marker of expertise? (e.g. there’s not a vast reserve of commercial positions for pure philosophers.)

I wonder if Lord Browne has heard about this yet.