Wikipedia trounces Britannica in blind taste test!

[Randy from Boise]

Mass Collaboration or Mass Amateurism?

http://massamateurism.blogspot.com/p/sy ... l#_ftnref2

"One explanation for the success of Wikipedia can be found in the altruism of individuals who come together around themes they share the same passion for, and it is expected they have remarkable knowledge on these. In other words, in the case studied, mass collaboration seems self-organized, leading to an organization of self-assessment and self-correction among peers which produces impressive results, against all odds."

Discuss.

[The Devil's Advocate]

Mmmmm, it tastes like porn.

[Midsize Jake]

Is he comparing the English Wikipedia to Britannica, or the Portuguese Wikipedia? The author of the dissertation is Portuguese, and so are all of the experts he chose (apparently by himself, I might add). I can see how Portuguese experts would prefer the Portuguese Wikipedia to Britannica in English... Is there a Portuguese edition of Britannica, then? I'm pretty sure there isn't, but in any event, I don't see where he makes this clear or even mentions it.

And of course he was no shrinking violet when it came time to promote his new dissertation on WP.

Honestly, I can't even tell if this is a flawed study or not, though I have to say it doesn't look so good based on what I'm seeing.

[thekohser]

I note a couple of things.

First, the study seems to admit that "more words" equals "better article" in the minds of the judges. That seems a bit troubling.

Second, the study randomly selected about 25x more Wikipedia articles than Britannica articles, so that Wikipedia articles could be "matched" to corresponding Britannica articles. This generally would mean that Britannica (which curates its content far more carefully than Wikipedia) would be pitting "all" of its articles against only the "selected few" more noteworthy Wikipedia articles.

In other words, Britannica would be putting its article about Neptune up against Wikipedia's article about Neptune (T-H-L); but Wikipedia's article about Neptune Avenue (IND Culver Line) (T-H-L) (a lovely but completely observational bit of original research) would never enter the study, because Britannica isn't so interested in having an article about every rail stop in New York City.

All that being said, I'll be the first to say that Wikipedia's breadth is a lot more fun for the reader than Britannica ever was or will be. I mean, would Britannica ever be able to tell me that the protagonist in Wetlands secretly rams the pedal of her hospital bed into her anus and immediate emergency surgery has to be carried out to prevent extreme blood loss? I hardly think so!

[Randy from Boise]

I note a couple of things.

First, the study seems to admit that "more words" equals "better article" in the minds of the judges. That seems a bit troubling.

Second, the study randomly selected about 25x more Wikipedia articles than Britannica articles, so that Wikipedia articles could be "matched" to corresponding Britannica articles. This generally would mean that Britannica (which curates its content far more carefully than Wikipedia) would be pitting "all" of its articles against only the "selected few" more noteworthy Wikipedia articles.

In other words, Britannica would be putting its article about Neptune up against Wikipedia's article about Neptune (T-H-L); but Wikipedia's article about Neptune Avenue (IND Culver Line) (T-H-L) (a lovely but completely observational bit of original research) would never enter the study, because Britannica isn't so interested in having an article about every rail stop in New York City.

All that being said, I'll be the first to say that Wikipedia's breadth is a lot more fun for the reader than Britannica ever was or will be. I mean, would Britannica ever be able to tell me that the protagonist in Wetlands secretly rams the pedal of her hospital bed into her anus and immediate emergency surgery has to be carried out to prevent extreme blood loss? I hardly think so!

Would the methodology have been better to do the random selection from Britannica and then to match selected articles to the (vastly larger) Wikipedia?

As for the "necessarily unselected" WP articles, as long as we can agree on the premise that "something beats nothing," these are all default superiorities of WP to EB. Of course, we may agree to differ on whether vapid sports bios or pieces on Pokemon characters have greater than zero value...

RfB

[HRIP7]

I note a couple of things.

First, the study seems to admit that "more words" equals "better article" in the minds of the judges. That seems a bit troubling.

That was the thing that struck me most. They're judging quantity rather than quality.

[thekohser]

Would the methodology have been better to do the random selection from Britannica and then to match selected articles to the (vastly larger) Wikipedia?

That's essentially what happened, if you think about it. What this study says is that the average article from Britannica is probably inferior to Wikipedia's selected Britannica-worthy articles. At this point, I am not surprised that Wikipedia has so evolved.

On another token, though... I would say that if you were to read the Encyclopedia Britannica article about the band Aerosmith, the chances of you finding something like "Aerosmith, however, almost broke up after Tim Collins spread rumors that band members were deriding each other and that Tyler was being unfaithful to his wife and using drugs again during recording sessions in Miami", would be nearly nil; while the chances of you finding that on Wikipedia are nearly 100%. On Wikipedia, it's entirely unsourced, and it either violates the reputation of living person Tim Collins, or of living person Steven Tyler -- take your pick. Encyclopedia Britannica doesn't typically do gossip. For Wikipedia, it's their stock-in-trade.

So, it's sort of like imagining if the National Enquirer, Fox News, David Shankbone's blog, the Washington Post, and Scientific American all merged to form one new content site, that would "trounce" either the New York Times or Nature alone. I'll leave the thoughtful readers here to think about that for a few moments.

[Midsize Jake]

Would the methodology have been better to do the random selection from Britannica and then to match selected articles to the (vastly larger) Wikipedia?

That's essentially what happened, if you think about it. What this study says is that the average article from Britannica is probably inferior to Wikipedia's selected Britannica-worthy articles. At this point, I am not surprised that Wikipedia has so evolved.

Right - the article selection methodology isn't the problem at all, though limiting it to a handful of general topic areas is suspicious. The real problem is that this guy, who is already biased towards Wikipedia anyway, personally selected a group of Portuguese college professors to compare reference materials from two sources written in English. Surprise-surprise, they decided that the longer articles must be better than the shorter ones.

Always remember that judging Wikipedia on "accuracy" is a canard, a red herring. It's also self-serving for them, since lack of "accuracy" is a problem that they'll always claim can be solved by bringing in more users (though of course this is not actually true). Besides, any failure of attaining accuracy in a Wikipedia article that has an analagous Britannica article can easily be fixed by comparing it to the Britannica article directly, in most cases (i.e., cases where the Britannica article is already known to be accurate). The problems with Wikipedia's content go much deeper than this, even if you overlook the fact that it can be changed at any time with no pre-publication review.

[roger_pearse]

Mass Collaboration or Mass Amateurism?

http://massamateurism.blogspot.com/p/sy ... l#_ftnref2

"One explanation for the success of Wikipedia can be found in the altruism of individuals who come together around themes they share the same passion for, and it is expected they have remarkable knowledge on these. In other words, in the case studied, mass collaboration seems self-organized, leading to an organization of self-assessment and self-correction among peers which produces impressive results, against all odds."

Discuss.

The basis for this is that some unspecified experts compared the two articles blind.

Some might consider me an expert, on the articles I used to take an interest in. Those articles are all, one and all, shoddy pieces of work that betray the copy-and-paste by ignorant people, only casually interested in the topic (and worse, sometimes with an axe to grind as well).

It may be that the "university teachers" don't know the subject.

But if I were handed a "study" like that, with its obvious problem that the quality of evaluation is unspecified, I'd reject it and write "try again" on it.

[Peter Damian]

Because Britannica is much smaller in size (≈25 smaller), 6.382 articles are randomly drawn from Wikipedia, in order to achieve 245 pairs of articles present in both encyclopedias.

I'm finding this hard to square with my experience in my own field. Britannica is much better in any serious subject. But I note they made the selection from Wikipedia, not Britannica.

So, it's sort of like imagining if the National Enquirer, Fox News, David Shankbone's blog, the Washington Post, and Scientific American all merged to form one new content site, that would "trounce" either the New York Times or Nature alone. I'll leave the thoughtful readers here to think about that for a few moments.

Much better.

[Peter Damian]

And I’ll point you again to this, cough.

[EricBarbour]

This is completely, dead typical of most "academic papers" I've seen about Wikipedia. They cherry-pick the suppositions,
then massage the data to fit the suppositions. Not a word about WP, or EB, systemic biases and unique perversities. Nice plots
of article size or "quality", with "quality" being poorly defined, tells us nothing about Wikipedia's trustworthiness.

That's another aspect of Wikipedia, it magically generates bad research, even if the research is about Wikipedia itself. A PhD dissertation
that quotes from Clay Shirky can probably be safely dismissed. Also if you see footnotes like this one.

[5] Unfortunately, it was impossible to find experts available to grade this category (only 5% of the articles are graded), so the conclusions are about all the fields of knowledge except Geography.