Harassment Survey 2015

User avatar
Starke Hathaway
Critic
Posts: 155
kołdry
Joined: Thu May 28, 2015 10:19 pm
Wikipedia User: Starke Hathaway

Re: Harassment Survey 2015

Unread post by Starke Hathaway » Wed Feb 03, 2016 11:05 pm

SneakySasha wrote:
Earthy Astringent wrote:The amount of passive aggressive behavior is off the charts when a spat breaks out, and battlesides are quickly drawn and a moderator has to lock the topic,

If a someone makes an argument against the house POV, the house will hen-peck the dissenter if they fail to come around. And I intentionally used "hen peck" because once blood appears everyone wants to get a taste. On the rare occasion that the dissenter is able to prove they are "right" (borrowing from Graham's hierarchy of arguments), one of the Alphas will announce "I'm ignoring X!", and the greek chorus yells "me too!"
Although, I fail to see how this description is any different from Wikipedia.
When I first read this comment (hastily, I admit) I thought it was about Wikipedia.

User avatar
thekohser
Majordomo
Posts: 13410
Joined: Thu Mar 15, 2012 5:07 pm
Wikipedia User: Thekohser
Wikipedia Review Member: thekohser
Actual Name: Gregory Kohs
Location: United States
Contact:

Re: Harassment Survey 2015

Unread post by thekohser » Thu Feb 04, 2016 12:05 am

SneakySasha wrote:I'm wondering if any of you have ever actually spent any time on any forums where the majority of the users are women? 'Cause overall, it's much, much less of what I can only classify as a "war-like" environment.
Facebook has more adult women users than adult male users. It feels a little "war-like" to me oftentimes.
"...making nonsensical connections and culminating in feigned surprise, since 2006..."

User avatar
Demonology
Critic
Posts: 130
Joined: Sat Apr 11, 2015 4:25 am
Actual Name: Beatrix

Re: Harassment Survey 2015

Unread post by Demonology » Thu Feb 04, 2016 12:19 am

Drijfzand wrote:Here's your problem!
Nice detective work, Drijfzand! I found those percentages really hard to believe, but wikipediot incompetence on the other hand is definitely believable.
"Aurora borealis?? At this time of year, at this time of day, in this part of the country, localized entirely within your kitchen?!"

User avatar
thekohser
Majordomo
Posts: 13410
Joined: Thu Mar 15, 2012 5:07 pm
Wikipedia User: Thekohser
Wikipedia Review Member: thekohser
Actual Name: Gregory Kohs
Location: United States
Contact:

Re: Harassment Survey 2015

Unread post by thekohser » Thu Feb 04, 2016 12:35 am

Earthy Astringent wrote:All of the alleged problems with the survey aside, how much do you think it would cost the WMF to hire a professional to create, execute, and analyze a "harassment" survey?
Honestly, not a heck of a lot. One of the most expensive aspects of opinion survey data collection is in obtaining cooperative respondent sample. With the Wikimedia projects, the cooperative respondents are there on your website, every day. I'm sure you could find a reputable consultant with at least 10 years of scientific sampling experience who would run a legitimate survey about harassment for the Wikimedia Foundation for about $10,000 to $15,000.

Hell, in preparation for a talk that I was giving at a fuel merchants convention in New Jersey, I once ran a survey using respondents from Amazon's Mechanical Turk service. I was able to get about 75 respondents (granted, that's a tiny sample for quantitative work, but good enough for a conference slide or two), and I produced the following charts for my presentation:

Image

Total out-of-pocket cost? Eight dollars and fourteen cents.

Image

The survey took me about an hour to write and program and field, and the data analysis and construction of the presentation slides took me another two hours or so. If the Wikimedia Foundation were to hire me to conduct a legitimate, scientifically-designed population survey of its projects' readers and editors, I would charge them $10,000 for my professional time, since there would be no sample costs involved. Where we would get hung up is when I would insist that we randomly sample readers and editors, and they would say, "no, that's not as open and inclusive as possible", at which point I would tell them, "no, it isn't, but that's how grown-ups conduct scientifically-designed population surveys", and at that point they'd probably give the contract to someone else.


Anyway, since we seem to be pretty good at being able to advise the Wikimedia Foundation on how not to run their surveys, maybe it would help if we gave them constructive guidance on how to design their surveys in the future?
"...making nonsensical connections and culminating in feigned surprise, since 2006..."

SneakySasha
Contributor
Posts: 82
Joined: Wed Dec 09, 2015 8:47 am

Re: Harassment Survey 2015

Unread post by SneakySasha » Thu Feb 04, 2016 1:07 am

thekohser wrote:
SneakySasha wrote:I'm wondering if any of you have ever actually spent any time on any forums where the majority of the users are women? 'Cause overall, it's much, much less of what I can only classify as a "war-like" environment.
Facebook has more adult women users than adult male users. It feels a little "war-like" to me oftentimes.
And, see, I don't do Facebook!!! :B'

User avatar
trout
Regular
Posts: 487
Joined: Wed Aug 21, 2013 7:24 am
Wikipedia User: Don City Break

Re: Harassment Survey 2015

Unread post by trout » Thu Feb 04, 2016 4:59 am

thekohser wrote:Where we would get hung up is when I would insist that we randomly sample readers and editors, and they would say, "no, that's not as open and inclusive as possible", at which point I would tell them, "no, it isn't, but that's how grown-ups conduct scientifically-designed population surveys"
Clearly, with the large number of editors, unbiased random sampling of the Wikipedia editor population is the only realistic way they could get an accurate picture of harassment. So, possibly they are actually trying to get a particular result or something by using a self-selected sample like this. "Oh no, harassment is rife, we must spend more money on <something we were already planning to spend money on>" or something like that.

Edit: clarified what I was trying to say.

User avatar
Jim
Blue Meanie
Posts: 4955
Joined: Fri Sep 07, 2012 10:33 am
Wikipedia User: Begoon
Wikipedia Review Member: Jim
Location: NSW

Re: Harassment Survey 2015

Unread post by Jim » Thu Feb 04, 2016 8:07 am

Jim wrote:
thekohser wrote:What did the invitation say?
It's one thing to say you're conducting a survey across a probability sample, please participate.
It's another thing to say you're conducting a survey about the harassment we experience on Wikimedia projects, please participate.
It said the latter, from memory.
Found it: HarassmentSurvey2015_A.

The wording was:
We invite you to participate in a survey about online harassment on Wikimedia projects.

User avatar
Johnny Au
Habitué
Posts: 2620
Joined: Fri Jan 31, 2014 5:05 pm
Wikipedia User: Johnny Au
Actual Name: Johnny Au
Location: Toronto, Ontario, Canada

Re: Harassment Survey 2015

Unread post by Johnny Au » Thu Feb 04, 2016 10:42 pm

Wikipedia must be very toxic (though fortunately, I generally edit the less contentious articles).

The only harassment I had on Wikipedia came from fans of rival sports teams (and they get blocked very quickly) and I don't edit articles on rival sports teams.

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Fri Feb 05, 2016 2:32 am

Demonology wrote:
Drijfzand wrote:Here's your problem!
Nice detective work, Drijfzand! I found those percentages really hard to believe, but wikipediot incompetence on the other hand is definitely believable.
I was wrong unfortunately.. :crying:

Lets hope this one is correct:

The qualtrixs slider page says that the user has to move the slider slightly for the question to count as answered instead of skipped.

the raw data shows the number of responses for each type. The report uses those numbers to calculate the percentages.

Once you move the slider, you're counted, regardless of the value. And with the warning message given when you didn't answer all the questions, some users will touch each slider and set it to zero.

They should be able to fix it, they will have the answers of each participant. I wonder what they'll do with answers like 0.09; for some reason they set it to 2 decimal precision...

edit: The guy who reported the error on 3 nov was mistaken saying all values had to be non-zero, he didn't realise that the sliders he had skipped were "not answered" instead of zero. And when he moved them to a different value it worked, so his confusion is understandable.
Damn, after complaining that people accept unlikely data, I was willing to believe that hundreds of participants would rather submit wrong data than quit the survey... And not questioning that only one person reported it :facepalm:
Last edited by Drijfzand on Fri Feb 05, 2016 6:26 am, edited 1 time in total.
Tweaker in Metropolis

User avatar
thekohser
Majordomo
Posts: 13410
Joined: Thu Mar 15, 2012 5:07 pm
Wikipedia User: Thekohser
Wikipedia Review Member: thekohser
Actual Name: Gregory Kohs
Location: United States
Contact:

Re: Harassment Survey 2015

Unread post by thekohser » Fri Feb 05, 2016 4:30 am

Drijfzand wrote:I wonder what they'll do with answers like 0.09; for some reason they set it to 2 decimal precision...
Somebody probably told the WMF staff that two decimal places are more open and inclusive than just one, and certainly heaps better than zero.
"...making nonsensical connections and culminating in feigned surprise, since 2006..."

User avatar
thekohser
Majordomo
Posts: 13410
Joined: Thu Mar 15, 2012 5:07 pm
Wikipedia User: Thekohser
Wikipedia Review Member: thekohser
Actual Name: Gregory Kohs
Location: United States
Contact:

Re: Harassment Survey 2015

Unread post by thekohser » Fri Feb 05, 2016 1:56 pm

OMG...

As Drijfzand says above, if this is true, then sliding the slider bar from the "non-response" to "zero" position, actually counted every "zero" response as a "yes - some level of harassment of that type has been experienced by me".

The Wikimedia Foundation should just be blown up and started over.
"...making nonsensical connections and culminating in feigned surprise, since 2006..."

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Fri Feb 05, 2016 2:47 pm

... and mathematical proof added...

My job is done :B'
Tweaker in Metropolis

User avatar
Kingsindian
Habitué
Posts: 2593
Joined: Fri Nov 20, 2015 10:07 am
Wikipedia User: Kingsindian

Re: Harassment Survey 2015

Unread post by Kingsindian » Fri Feb 05, 2016 3:08 pm

I corrected a small typo in the math, but otherwise calculations look fine.

User avatar
Zoloft
Trustee
Posts: 14081
Joined: Wed Mar 14, 2012 11:54 pm
Wikipedia User: Stanistani
Wikipedia Review Member: Zoloft
Actual Name: William Burns
Nom de plume: William Burns
Location: San Diego
Contact:

Re: Harassment Survey 2015

Unread post by Zoloft » Fri Feb 05, 2016 4:27 pm

Every WMF report should have a section entitled "What is salvageable from this mess?"

My avatar is sometimes indicative of my mood:
  • Actual mug ◄
  • Uncle Cornpone
  • Zoloft bouncy pill-thing


User avatar
Cedric
Habitué
Posts: 1049
Joined: Fri Mar 16, 2012 3:01 am
Wikipedia User: Edeans
Wikipedia Review Member: Cedric
Actual Name: Eddie Singleton
Location: God's Ain Country

Re: Harassment Survey 2015

Unread post by Cedric » Sat Feb 06, 2016 4:13 am

Zoloft wrote:Every WMF report should have a section entitled "What is salvageable from this mess?"
I don't think that is sufficient. They should also be prefaced by "WARNING: CONTAINS HOT MESS AND FAIL."

User avatar
Vigilant
Sonny, I've got a whole theme park full of red delights for you.
Posts: 31777
Joined: Thu Mar 29, 2012 8:16 pm
Wikipedia User: Vigilant
Wikipedia Review Member: Vigilant

Re: Harassment Survey 2015

Unread post by Vigilant » Sat Feb 06, 2016 4:28 am

Cedric wrote:
Zoloft wrote:Every WMF report should have a section entitled "What is salvageable from this mess?"
I don't think that is sufficient. They should also be prefaced by "WARNING: CONTAINS HOT MESS AND FAIL."
FTFY

No historical evidence of even the slightest adulteration.
Hello, John. John, hello. You're the one soul I would come up here to collect myself.

User avatar
Kumioko
Muted
Posts: 6609
Joined: Sun Mar 03, 2013 2:36 am
Wikipedia User: Kumioko; Reguyla
Nom de plume: Persona non grata

Re: Harassment Survey 2015

Unread post by Kumioko » Sat Feb 06, 2016 11:36 pm

Vigilant wrote:
Cedric wrote:
Zoloft wrote:Every WMF report should have a section entitled "What is salvageable from this mess?"
I don't think that is sufficient. They should also be prefaced by "WARNING: CONTAINS HOT MESS AND FAIL."
FTFY

No historical evidence of even the slightest adulteration.
First its important to note that the WMF doesn't care about anything on the projects or the communities other than generating revenue. As such, surveys like this are a waste of time because the WMF does not care and does not want to do anything about them. Even if these survey's named specifically who the problems were, they still wouldn't do anything. What these surveys do for the WMF is allow them to tell potential donors that they are doing surveys on harassment, editing, favorite food, etc. and then that makes it look like they are a proactive organization.

Of course those of us in the community that actually see the WMF know better, the donors don't edit and therefore do not know and sign the big checks.

User avatar
Randy from Boise
Been Around Forever
Posts: 12237
Joined: Sun Mar 18, 2012 2:32 am
Wikipedia User: Carrite
Wikipedia Review Member: Timbo
Actual Name: Tim Davenport
Nom de plume: T. Chandler
Location: Boise, Idaho

Re: Harassment Survey 2015

Unread post by Randy from Boise » Sun Feb 07, 2016 2:47 am

Kumioko wrote: First its important to note that the WMF doesn't care about anything on the projects or the communities other than generating revenue. As such, surveys like this are a waste of time because the WMF does not care and does not want to do anything about them.
That's simply not true, you are overlooking the touchie-feelie quasi-religious aspect. "The only good employees are those who have community ties," and all that. Certainly there is a tension between their fundraising and empire-building on the one hand and the tradition of volunteerism on the other, but it's absolutely wrong to state or intimate that the WMF doesn't care about the projects. They are paternalistic and condescending and dismissive, mind you, but they also want to hold hands and sing Kumbaya around the campfire with the world in all its blissful, friendly, diversity...

It's a huge fuckin' weird streak that can't be ignored, no matter how bitter you personally are about feeling shat upon.

RfB

User avatar
lilburne
Habitué
Posts: 4446
Joined: Thu Mar 15, 2012 6:18 pm
Wikipedia User: Nastytroll
Wikipedia Review Member: Lilburne

Re: Harassment Survey 2015

Unread post by lilburne » Sun Feb 07, 2016 10:07 am

Given the mentality of the wikicomedians it is likely that they believe that Pepperland has been attacked by Blue Meanies and all it needs is for the Yellow Submarine in the shape of the WMF to turn up and everything will be OK.
They have been inserting little memes in everybody's mind
So Google's shills can shriek there whenever they're inclined

User avatar
Jim
Blue Meanie
Posts: 4955
Joined: Fri Sep 07, 2012 10:33 am
Wikipedia User: Begoon
Wikipedia Review Member: Jim
Location: NSW

Re: Harassment Survey 2015

Unread post by Jim » Sun Feb 07, 2016 10:13 am

lilburne wrote:Given the mentality of the wikicomedians it is likely that they believe that Pepperland has been attacked by Blue Meanies and all it needs is for the Yellow Submarine in the shape of the WMF to turn up and everything will be OK.
Phrase coined. Please wait while meme initiates...

I want "Blue Meanie" above my avatar.

User avatar
Zoloft
Trustee
Posts: 14081
Joined: Wed Mar 14, 2012 11:54 pm
Wikipedia User: Stanistani
Wikipedia Review Member: Zoloft
Actual Name: William Burns
Nom de plume: William Burns
Location: San Diego
Contact:

Re: Harassment Survey 2015

Unread post by Zoloft » Sun Feb 07, 2016 10:21 am

Jim wrote:
lilburne wrote:Given the mentality of the wikicomedians it is likely that they believe that Pepperland has been attacked by Blue Meanies and all it needs is for the Yellow Submarine in the shape of the WMF to turn up and everything will be OK.
Phrase coined. Please wait while meme initiates...

I want "Blue Meanie" above my avatar.
:sparkles:

My avatar is sometimes indicative of my mood:
  • Actual mug ◄
  • Uncle Cornpone
  • Zoloft bouncy pill-thing


User avatar
Jim
Blue Meanie
Posts: 4955
Joined: Fri Sep 07, 2012 10:33 am
Wikipedia User: Begoon
Wikipedia Review Member: Jim
Location: NSW

Re: Harassment Survey 2015

Unread post by Jim » Sun Feb 07, 2016 10:24 am

Zoloft wrote: :sparkles:
:B'

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Sun Feb 07, 2016 2:04 pm

Kolbe has written a draft for the Signpost.
The survey results have been discussed on both Jimmy Wales' talk page and on the survey's talk page on Meta. Volunteers have questioned some of the results and alleged design defects in the survey, and WMF staff have responded.
Staff hasn't yet responded to the claim that they are using the wrong figures. Well, apart from admitting, there little they can say.. The statistics listed in raw data are what you would get from sliders without snap-to-grid (with snap-to-grid the number of responses per value present are listed). The response count in that data is the total number of responses. The average and sdev values prove that the data contains responses smaller than 1.

Wonder what they will do. I only hope they don't drop it citing "unspecified issues that raised doubt about the validity of the data".
If they bring out a revised version, quite a few people will look gullible for believing the first one.
If they keep it things could get interesting. With wikipedia you never know, they'll probably quote WP:NOR.

They promised when the preliminary report came out that they would have more within a week. The one bit of data they haven't used yet is the values entered in the question. Judging by the raw data stats those figures are also highly unlikely. Like 25 people hacked 80 times each. Could be due to using sliders.
Btw, how many "jokers" deliberately entering fake data would one expect in a typical survey?
Tweaker in Metropolis

User avatar
thekohser
Majordomo
Posts: 13410
Joined: Thu Mar 15, 2012 5:07 pm
Wikipedia User: Thekohser
Wikipedia Review Member: thekohser
Actual Name: Gregory Kohs
Location: United States
Contact:

Re: Harassment Survey 2015

Unread post by thekohser » Sun Feb 07, 2016 2:21 pm

Drijfzand wrote:Btw, how many "jokers" deliberately entering fake data would one expect in a typical survey?
In my experience, in B2B surveys, maybe 1% to 3%. In consumer surveys, maybe 3% to 5%. In Wikipedia-related surveys, I would guess that it's 5% to 7%.
"...making nonsensical connections and culminating in feigned surprise, since 2006..."

User avatar
lilburne
Habitué
Posts: 4446
Joined: Thu Mar 15, 2012 6:18 pm
Wikipedia User: Nastytroll
Wikipedia Review Member: Lilburne

Re: Harassment Survey 2015

Unread post by lilburne » Sun Feb 07, 2016 2:52 pm

Drijfzand wrote: They promised when the preliminary report came out that they would have more within a week. The one bit of data they haven't used yet is the values entered in the question. Judging by the raw data stats those figures are also highly unlikely. Like 25 people hacked 80 times each.
To be perfectly honest I couldn't care less. Whether the report is accurate or not. Whether they draw the right or wrong conclusion from it or not. None of it matters the WMF won't do anything, the wikicomedian community will whine and bitch as they always do. Nothing will change except that more and more people will become disillusioned from the site and the regulars will become more insular and suspicious of newcomers.

Its a win win win all round.
They have been inserting little memes in everybody's mind
So Google's shills can shriek there whenever they're inclined

User avatar
Randy from Boise
Been Around Forever
Posts: 12237
Joined: Sun Mar 18, 2012 2:32 am
Wikipedia User: Carrite
Wikipedia Review Member: Timbo
Actual Name: Tim Davenport
Nom de plume: T. Chandler
Location: Boise, Idaho

Re: Harassment Survey 2015

Unread post by Randy from Boise » Sun Feb 07, 2016 3:04 pm

thekohser wrote:
Drijfzand wrote:Btw, how many "jokers" deliberately entering fake data would one expect in a typical survey?
In my experience, in B2B surveys, maybe 1% to 3%. In consumer surveys, maybe 3% to 5%. In Wikipedia-related surveys, I would guess that it's 5% to 7%.
Greg:— Just as a general query — not that WMF would ever admit that even a single troll would infiltrate their thoughtful, friendly surveys — is it possible to eliminate most or all of the "jokers" with a trap question? For example, in the list of types of harassment experienced by inserting something nonsensical, like "hot-foot" and then tossing all responses of anyone surveyed who selected YES or assigned a number for that?

It seems like one could trap off about half the trolls that way, or more.

Is this sort of thing done in real world surveyland?

RfB

User avatar
Moral Hazard
Super Genius
Posts: 3401
Joined: Tue Feb 17, 2015 4:46 pm
Wikipedia User: Kiefer.Wolfowitz
Nom de plume: Kiefer Wolfowitz
Contact:

Re: Harassment Survey 2015

Unread post by Moral Hazard » Sun Feb 07, 2016 6:27 pm

Survey methodologists have questions that identify likely suspects who enter what the survey wants to hear. There are specialized sources for on-line surveys.
Kiefer.Wolfowitz (T-C-L)
“Arguing with anonymous strangers on the Internet is a sucker's game because they almost always turn out to be—or to be indistinguishable from—self-righteous sixteen-year-olds possessing infinite amounts of free time.”
Neal Stephenson (T-H-L) Cryptonomicon

User avatar
Poetlister
Genius
Posts: 25599
Joined: Wed Jan 02, 2013 8:15 pm
Nom de plume: Poetlister
Location: London, living in a similar way
Contact:

Re: Harassment Survey 2015

Unread post by Poetlister » Sun Feb 07, 2016 10:16 pm

Don't forget that Wikiworld has throngs of sophisticated trolls who would probably see through such traps.
"The higher we soar the smaller we appear to those who cannot fly" - Nietzsche

ArmasRebane
Gregarious
Posts: 995
Joined: Wed Nov 18, 2015 7:04 pm

Re: Harassment Survey 2015

Unread post by ArmasRebane » Mon Feb 08, 2016 4:12 pm

Randy from Boise wrote:
thekohser wrote:
Drijfzand wrote:Btw, how many "jokers" deliberately entering fake data would one expect in a typical survey?
In my experience, in B2B surveys, maybe 1% to 3%. In consumer surveys, maybe 3% to 5%. In Wikipedia-related surveys, I would guess that it's 5% to 7%.
Greg:— Just as a general query — not that WMF would ever admit that even a single troll would infiltrate their thoughtful, friendly surveys — is it possible to eliminate most or all of the "jokers" with a trap question? For example, in the list of types of harassment experienced by inserting something nonsensical, like "hot-foot" and then tossing all responses of anyone surveyed who selected YES or assigned a number for that?

It seems like one could trap off about half the trolls that way, or more.

Is this sort of thing done in real world surveyland?

RfB
In a random example, Microsoft's feedback surveys often contain a question that just says "Answer with option (c)" or similar to catch people answering randomly or without reading questions; presumably they exclude these surveys from the final dataset entirely.

It's essentially the non-school equivalent of those "read all the directions" tests that had you do a bunch of irrelevant stuff unless you actually did read the directions, which said not to do a thing at the very bottom.

User avatar
Jim
Blue Meanie
Posts: 4955
Joined: Fri Sep 07, 2012 10:33 am
Wikipedia User: Begoon
Wikipedia Review Member: Jim
Location: NSW

Re: Harassment Survey 2015

Unread post by Jim » Mon Feb 08, 2016 4:54 pm

ArmasRebane wrote:those "read all the directions" tests that had you do a bunch of irrelevant stuff unless you actually did read the directions, which said not to do a thing at the very bottom.
Heh. I dimly remember falling for one of those. As I recall, everyone else in the class fell for it too (bar maybe one or two, it's long ago.)

It was a boys school, though, so that could just be "statistical evidence" that males are not overly inclined to RTFM before diving in...

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Mon Feb 08, 2016 11:58 pm

I am sad to let you know that Luis Villa, our lead for the Community
Engagement department, will be leaving the Wikimedia Foundation.
..
Maggie Dennis will step in as the interim director for the CE team
effective immediately. Her deep community background, passion for our
mission, and outstanding teamwork are great assets in this transition. She
will also continue to serve as Director of Support and Safety. Maggie is a
respected leader, colleague, and community member. I am confident she will
bring critical insights, especially now as we plan for our next year.
http://www.gossamer-threads.com/lists/w ... ion/677493
Critical lack of insights more likely. Support and Safety is the department responsible for the survey report...
Tweaker in Metropolis

User avatar
Zoloft
Trustee
Posts: 14081
Joined: Wed Mar 14, 2012 11:54 pm
Wikipedia User: Stanistani
Wikipedia Review Member: Zoloft
Actual Name: William Burns
Nom de plume: William Burns
Location: San Diego
Contact:

Re: Harassment Survey 2015

Unread post by Zoloft » Tue Feb 09, 2016 10:39 am

Drijfzand wrote:
I am sad to let you know that Luis Villa, our lead for the Community
Engagement department, will be leaving the Wikimedia Foundation.
..
Maggie Dennis will step in as the interim director for the CE team
effective immediately. Her deep community background, passion for our
mission, and outstanding teamwork are great assets in this transition. She
will also continue to serve as Director of Support and Safety. Maggie is a
respected leader, colleague, and community member. I am confident she will
bring critical insights, especially now as we plan for our next year.
http://www.gossamer-threads.com/lists/w ... ion/677493
Critical lack of insights more likely. Support and Safety is the department responsible for the survey report...
Aka Moonriddengirl (T-C-L).

My avatar is sometimes indicative of my mood:
  • Actual mug ◄
  • Uncle Cornpone
  • Zoloft bouncy pill-thing


User avatar
Randy from Boise
Been Around Forever
Posts: 12237
Joined: Sun Mar 18, 2012 2:32 am
Wikipedia User: Carrite
Wikipedia Review Member: Timbo
Actual Name: Tim Davenport
Nom de plume: T. Chandler
Location: Boise, Idaho

Re: Harassment Survey 2015

Unread post by Randy from Boise » Tue Feb 09, 2016 12:18 pm

Zoloft wrote:
Drijfzand wrote:
I am sad to let you know that Luis Villa, our lead for the Community
Engagement department, will be leaving the Wikimedia Foundation.
..
Maggie Dennis will step in as the interim director for the CE team
effective immediately. Her deep community background, passion for our
mission, and outstanding teamwork are great assets in this transition. She
will also continue to serve as Director of Support and Safety. Maggie is a
respected leader, colleague, and community member. I am confident she will
bring critical insights, especially now as we plan for our next year.
http://www.gossamer-threads.com/lists/w ... ion/677493
Critical lack of insights more likely. Support and Safety is the department responsible for the survey report...
Aka Moonriddengirl (T-C-L).
I like Maggie Dennis a lot. She was the top volunteer working in the field of copyright and is hard working, open-minded, and intelligent. It always struck me as one of the great crimes of WMF that they pulled her from the volunteer editorial ranks (where she remains desperately needed) and made her a paid WMF functionary. On the other hand, if anyone is capable of making a successful and effective transition from one milieu to the other, it would be she.

RfB

User avatar
Kingsindian
Habitué
Posts: 2593
Joined: Fri Nov 20, 2015 10:07 am
Wikipedia User: Kingsindian

Re: Harassment Survey 2015

Unread post by Kingsindian » Tue Feb 09, 2016 1:06 pm

I have had some conversations with Moonriddengirl on Wikipedia about copyright and close paraphrasing. She was quite helpful and also helped me out with rephrasing some text I had written.

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Tue Feb 09, 2016 3:36 pm

Randy from Boise wrote:
Zoloft wrote:
Drijfzand wrote:
I am sad to let you know that Luis Villa, our lead for the Community
Engagement department, will be leaving the Wikimedia Foundation.
..
Maggie Dennis will step in as the interim director for the CE team
effective immediately. Her deep community background, passion for our
mission, and outstanding teamwork are great assets in this transition. She
will also continue to serve as Director of Support and Safety. Maggie is a
respected leader, colleague, and community member. I am confident she will
bring critical insights, especially now as we plan for our next year.
http://www.gossamer-threads.com/lists/w ... ion/677493
Critical lack of insights more likely. Support and Safety is the department responsible for the survey report...
Aka Moonriddengirl (T-C-L).
I like Maggie Dennis a lot. She was the top volunteer working in the field of copyright and is hard working, open-minded, and intelligent. It always struck me as one of the great crimes of WMF that they pulled her from the volunteer editorial ranks (where she remains desperately needed) and made her a paid WMF functionary. On the other hand, if anyone is capable of making a successful and effective transition from one milieu to the other, it would be she.

RfB
Ok, maybe I was a bit harsh, but I'm getting tired of waiting for a response. Probably not uncommon when dealing with the WMF...
Tweaker in Metropolis

User avatar
lilburne
Habitué
Posts: 4446
Joined: Thu Mar 15, 2012 6:18 pm
Wikipedia User: Nastytroll
Wikipedia Review Member: Lilburne

Re: Harassment Survey 2015

Unread post by lilburne » Tue Feb 09, 2016 3:41 pm

Drijfzand wrote:
I am sad to let you know that Luis Villa, our lead for the Community
Engagement department, will be leaving the Wikimedia Foundation.
..
Maggie Dennis will step in as the interim director for the CE team
effective immediately. Her deep community background, passion for our
mission, and outstanding teamwork are great assets in this transition. She
will also continue to serve as Director of Support and Safety. Maggie is a
respected leader, colleague, and community member. I am confident she will
bring critical insights, especially now as we plan for our next year.
They have been inserting little memes in everybody's mind
So Google's shills can shriek there whenever they're inclined

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Tue Feb 09, 2016 3:58 pm

She does indeed reply...
Hi. :) This is my volunteer account, so I wouldn't talk work here, but beyond that I'm afraid don't know. I did not produce the report. The harassment project overall is being project managed by Patrick Earley. If you ask him, I'm sure he can figure out who to talk to. --Moonriddengirl (talk) 15:17, 9 February 2016 (UTC)
https://en.wikipedia.org/wiki/User_talk:Moonriddengirl

Earley's last post with his WMF account was on the 3rd. oh fuck it, got better things to do.
Tweaker in Metropolis

User avatar
thekohser
Majordomo
Posts: 13410
Joined: Thu Mar 15, 2012 5:07 pm
Wikipedia User: Thekohser
Wikipedia Review Member: thekohser
Actual Name: Gregory Kohs
Location: United States
Contact:

Re: Harassment Survey 2015

Unread post by thekohser » Tue Feb 09, 2016 8:40 pm

Drijfzand wrote:Earley's last post with his WMF account was on the 3rd. oh fuck it, got better things to do.
I'm sure that Neotarf will stay on the task, on your behalf.

:D
"...making nonsensical connections and culminating in feigned surprise, since 2006..."

User avatar
Kumioko
Muted
Posts: 6609
Joined: Sun Mar 03, 2013 2:36 am
Wikipedia User: Kumioko; Reguyla
Nom de plume: Persona non grata

Re: Harassment Survey 2015

Unread post by Kumioko » Tue Feb 09, 2016 9:07 pm

Randy from Boise wrote:
Kumioko wrote: First its important to note that the WMF doesn't care about anything on the projects or the communities other than generating revenue. As such, surveys like this are a waste of time because the WMF does not care and does not want to do anything about them.
That's simply not true, you are overlooking the touchie-feelie quasi-religious aspect. "The only good employees are those who have community ties," and all that. Certainly there is a tension between their fundraising and empire-building on the one hand and the tradition of volunteerism on the other, but it's absolutely wrong to state or intimate that the WMF doesn't care about the projects. They are paternalistic and condescending and dismissive, mind you, but they also want to hold hands and sing Kumbaya around the campfire with the world in all its blissful, friendly, diversity...

It's a huge fuckin' weird streak that can't be ignored, no matter how bitter you personally are about feeling shat upon.

RfB
I guess we'll have to agree to disagree. Its been my experience that the WMF triesto talk a good game but really doesn't care what goes on in the projects. Sure they want to say they support them, but all they do is keep the servers running (and with a lot of problems it seems) and cause hate and discontent. I don't deny that there are some in the WMF that want to see the projects succeed like Maggie Dennis but that group is very very small. The vast majority of employees won't even edit because they are afraid if something happens on project it could affect their job. So if they aren't already established before they get the job, they are unlikely to edit and, I have been told by a few, that its actually discouraged.

I also don't think its a bad thing that some don't edit actively. They should all have accounts and understand the basics, but having some folks who are outside the social circle and can look on a situation without bias or preconceived notions or pet projects in mind is actually a good thing.

Also, just to clarify, I am not bitter at the WMF. I am bitter about the Arbcom and some of the admins who think policy doesn't apply to them. The only the the WMF has done wrong on that aspect is let them get away with it.

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Wed Feb 10, 2016 2:00 am

I do feel sorry for the WMF staff when I read many of the comments on the Village Pump and other pages. It's one thing to do it here or on another off-wiki site, quite another to do it on the pages where you communicate with them...

Not a job I would want, WMF employee dealing with the "community".
Tweaker in Metropolis

User avatar
Vigilant
Sonny, I've got a whole theme park full of red delights for you.
Posts: 31777
Joined: Thu Mar 29, 2012 8:16 pm
Wikipedia User: Vigilant
Wikipedia Review Member: Vigilant

Re: Harassment Survey 2015

Unread post by Vigilant » Wed Feb 10, 2016 2:15 am

Drijfzand wrote:I do feel sorry for the WMF staff when I read many of the comments on the Village Pump and other pages. It's one thing to do it here or on another off-wiki site, quite another to do it on the pages where you communicate with them...

Not a job I would want, WMF employee dealing with the "community".
You have to admit that they've set the tone for the last five years as arrogant assholes who just won't listen to anyone.
This might be acceptable were it not for the flagrant incompetence which has mired the entire organization.
Hello, John. John, hello. You're the one soul I would come up here to collect myself.

User avatar
Kumioko
Muted
Posts: 6609
Joined: Sun Mar 03, 2013 2:36 am
Wikipedia User: Kumioko; Reguyla
Nom de plume: Persona non grata

Re: Harassment Survey 2015

Unread post by Kumioko » Wed Feb 10, 2016 5:39 pm

Drijfzand wrote:I do feel sorry for the WMF staff when I read many of the comments on the Village Pump and other pages. It's one thing to do it here or on another off-wiki site, quite another to do it on the pages where you communicate with them...

Not a job I would want, WMF employee dealing with the "community".
I have to be honest, I don't really feel too sorry for them. They have earned the comments and condemnation from the community through ineptitude, complacency and a negative attitude towards the community themselves. Much of the problems we see in the social and cultural problems on the sites are the direct result of the WMF either doing something that was counter to the success of the project or not doing something to make them better. Its all about raising money to them, not building an encyclopedia.

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Thu Feb 11, 2016 1:14 am

Vigilant wrote:
Drijfzand wrote:I do feel sorry for the WMF staff when I read many of the comments on the Village Pump and other pages. It's one thing to do it here or on another off-wiki site, quite another to do it on the pages where you communicate with them...

Not a job I would want, WMF employee dealing with the "community".
You have to admit that they've set the tone for the last five years as arrogant assholes who just won't listen to anyone.
This might be acceptable were it not for the flagrant incompetence which has mired the entire organization.
Could be, I don't know enough of the past history to judge. I read a lot of criticism about their software development, things either not working or not asked for by the community. Never tried any of them (visual editor etc..), apart from the article feedback tool, which was removed because the community didn't like it. Too much work to patrol and the feedback being "completely useless" were the objections, if I remember correctly.

Answer from Patrick Earley received btw:
Prevalence, we are looking at this issue, as well as working on a few more visualizations to include in our updated release as announced. Thank you for pointing this out, and engaging with the data. We will update the Talk page as soon as we can. Best, Patrick Earley (WMF) (talk) 18:41, 10 February 2016 (UTC)
Tweaker in Metropolis

User avatar
Poetlister
Genius
Posts: 25599
Joined: Wed Jan 02, 2013 8:15 pm
Nom de plume: Poetlister
Location: London, living in a similar way
Contact:

Re: Harassment Survey 2015

Unread post by Poetlister » Thu Feb 11, 2016 12:45 pm

Drijfzand wrote:the article feedback tool, which was removed because the community didn't like it.
It's difficult to believe that the real reason was community reaction. Don't forget, the WMF invented Superprotect rather than allow the community to stop VE being the default.
"The higher we soar the smaller we appear to those who cannot fly" - Nietzsche

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Sat Feb 13, 2016 11:05 am

Oh My God!

They found a way to avoid having to admit their blunder!!!

They don't mention the number of people who reported each type of harassment anymore, they simply give the averages... Which are completely misleading btw, from the standard deviation it was clear that for revenge porn and hacking there were 15 to 30 people reporting 70 or more experiences each.

https://upload.wikimedia.org/wikipedia/ ... Report.pdf
Tweaker in Metropolis

User avatar
Randy from Boise
Been Around Forever
Posts: 12237
Joined: Sun Mar 18, 2012 2:32 am
Wikipedia User: Carrite
Wikipedia Review Member: Timbo
Actual Name: Tim Davenport
Nom de plume: T. Chandler
Location: Boise, Idaho

Re: Harassment Survey 2015

Unread post by Randy from Boise » Sat Feb 13, 2016 12:06 pm

Drijfzand wrote:Oh My God!

They found a way to avoid having to admit their blunder!!!

They don't mention the number of people who reported each type of harassment anymore, they simply give the averages... Which are completely misleading btw, from the standard deviation it was clear that for revenge porn and hacking there were 15 to 30 people reporting 70 or more experiences each.

https://upload.wikimedia.org/wikipedia/ ... Report.pdf
On page 19 there are two separate sets of numbers for "stalking" on each line.

"Stalking" is ill-defined, in any event — it doesn't mean to a Wikipedian what it means to a normal person. It is a form of editing harassment, following a person from one edit to another via their edit history, not physical show-up-at-your-house-and-follow-you stalking...

The results are muddied by their insistence on making "other" genders a discrete category. I suspect that anyone really trolling the survey with silly answers would have selected "other" for their gender, since it is the silliest category. Again, this category does not include transexual men or transexual women — who were each assigned their own categories, which rounded to zero.

RfB

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Mon Feb 15, 2016 4:59 pm

Drijfzand wrote:Oh My God!

They found a way to avoid having to admit their blunder!!!

They don't mention the number of people who reported each type of harassment anymore, they simply give the averages... Which are completely misleading btw, from the standard deviation it was clear that for revenge porn and hacking there were 15 to 30 people reporting 70 or more experiences each.

https://upload.wikimedia.org/wikipedia/ ... Report.pdf
These people have more luck than they deserve. They count the wrong results, have to fix it, but thanks to their amateurish setup of the survey, those wrong results have averages that look enough like the results they were supposed to publish that they don't have to admit their mistake. They post the averages, which don't make sense as averages, but most people will look at the graph and interpret it as the prevalence of each type. It could even be fairly accurate, or not; we'll never know, because they will never give out the correct figures.

It's all thanks to the large number of zero values included in the raw data: with about 700 zero responses, and the number of (non-zero) results much smaller for most types, the average will be fairly proportional to the number of answers:
revenge porn: 740, threats of violence: 795
assuming users entered identical values
ratio of averages: (95/795)/(40/740) = 2.210
ratio of reported cases: 95/40=2.375
types with more responses will also have fewer zero's, so the ratio of averages will actually be higher than 2.21

It's still a worthless report, because there's no way of knowing how well it matches the real figures, and they are bloody lucky that the users entered inflated values due to the bad design of the survey (the sliders from 0 to 100), or the proportion between for example hacking (2040 incidents, with 14 people responsible for at least 1000 of them) and name calling (16235) would have been much larger.
(not that I believe that for every 8 persons being called names, one person is hacked, but it seems to be what people reported. the numbers for observed hacking are also inflated imo: of 1300 people, 18 witnessed hacking often, 64 occasionally: really?? Where are those cases?
Edit: woops, there is a way to know how well the numbers fit, if we assume that the ratios in the answering all group are the same as in the group that skipped N/A questions. For some scale factor, the results will be too big or small by a factor:
vandalism 1.19
trolling flam 1.17
name calling 1.02
discrimination 1.11
stalking 1.04
doxxing 0.83
threats 1.06
impersonation 0.87
hacking 0.86
revenge porn 1.20

smallest values are most susceptible to parameter changes, (the number of people who skipped N/A questions)

Anyway, there's no justice in the world! :angry:
Last edited by Drijfzand on Mon Feb 15, 2016 8:48 pm, edited 1 time in total.
Tweaker in Metropolis

User avatar
Randy from Boise
Been Around Forever
Posts: 12237
Joined: Sun Mar 18, 2012 2:32 am
Wikipedia User: Carrite
Wikipedia Review Member: Timbo
Actual Name: Tim Davenport
Nom de plume: T. Chandler
Location: Boise, Idaho

Re: Harassment Survey 2015

Unread post by Randy from Boise » Mon Feb 15, 2016 5:31 pm

I think we can all agree that this report has fundamental problems coming out of the structure. We'll see if they do better next time.

RfB

User avatar
thekohser
Majordomo
Posts: 13410
Joined: Thu Mar 15, 2012 5:07 pm
Wikipedia User: Thekohser
Wikipedia Review Member: thekohser
Actual Name: Gregory Kohs
Location: United States
Contact:

Re: Harassment Survey 2015

Unread post by thekohser » Mon Feb 15, 2016 7:06 pm

Randy from Boise wrote:We'll see if they do better next time.
:rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :blink: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl: :rotfl:
"...making nonsensical connections and culminating in feigned surprise, since 2006..."

User avatar
Drijfzand
Critic
Posts: 169
Joined: Sat Oct 03, 2015 12:33 pm
Location: Belgium

Re: Harassment Survey 2015

Unread post by Drijfzand » Wed Feb 17, 2016 9:30 am

thekohser wrote:
Drijfzand wrote:Earley's last post with his WMF account was on the 3rd. oh fuck it, got better things to do.
I'm sure that Neotarf will stay on the task, on your behalf.

:D
Checked her WP account today. And then looked up her name here. That explains a lot, I guess I was the only one who didn't know her yet...
She posted a comment:
The Foundation delivered the results of the survey on the date promised, along with the data dump. Even though there were obvious initial questions about the results, it was clear from the "preliminary" label on the survey that they understood what they had, and there was no attempt to hide or whitewash anything. Compare this to the 2012 editor survey. It seems odd to thank someone for performing their job in a professional, competent, and transparent manner, but thank you, WMF.

The Foundation has taken a hit lately in its relationship with the volunteer community, and not just with implementation of software products. There has been much skepticism surrounding the WMF commitment to the principles of privacy and non-discrimination and whether these sentiments are mere lip service or whether the Foundation intends to actually do anything about them. Much has yet to be seen, but this is a step in the right direction towards regaining the trust of the community, and in being willing to partner with the volunteer community in working towards mutual goals.

Wikimedians will no longer speak out publicly about harassment. They will only do so in the context of an anonymous survey.

The usual suspects (WP:BADSITES) showed up on this thread, having organized off-wiki, and predictably, once again made a collegial discussion impossible.

The arbitration committee has frequently been in collusion with WP:BADSITES in enabling harassment. I can't really get a take on the individual arbitrators who commented here, whether they believe the committee to be qualified to recognize and deal with harassment and discrimination, or whether they wish the Foundation to take a larger role. I looked at the case they cited, and didn't recognize any discrimination/harassment issues, although it did used to take me 12-15 hours to go through a case properly--time I don't have right now.

So what next? Does the survey go in a drawer, to be forgotten while the Safety team reorganizes itself, or will there be next steps? —Neotarf (talk) 20:23, 16 February 2016 (UTC)
Tweaker in Metropolis

Post Reply