Harassment Survey 2015

[Randy from Boise]

Wow, this figures.

An article about harassment of Wikipedians (particularly women) is published and brought up on Jimbo's talk page.

Various editors refute the validity of these claims, so some editors (including, for disclosure's sake, myself) respond on the talk page to say that we know from experience that this is a serious issue.

People decide that our experiences are not valid because we could not "prove" they happened (generally because they were suppressed or happened offwiki) or we were "special cases" being arbitrators/administrators/etc. (who apparently deserve to be harassed?)

A report comes out saying that harassment is a serious issue (and particularly an issue against minorities), and in this thread people decide to dismiss it as "whining," "propaganda," etc. Great stuff.

I'd like to publicly apologize for using the verb "whine" in my initial post. That was inappropriate, considering the enormity of the subject matter.

I missed GW's comment. For myself, I absolutely do not apologize for using the word "propaganda" in connection with this structurally lame and presentationally vapid set of PowerPoint slides represented as a "report." It is the amateur hour incarnate, which is par for the course for WMF.

Do not mischaracterize this perspective as dismissive of the actual problem of harassment. I have done no such thing and it is intellectually dishonest to imply otherwise, as you have done.

RfB

My other quick comment is that this report appears to be a propaganda document rather than a scientific study, as every page is marred by a multi-colored graph of some sort. It is more akin to a PowerPoint presentation than a serious study.

[Randy from Boise]

The most informative part of the survey on the amounts of harassment experienced by gender and cultural diversity are really these two graphics. It's interesting they didn't reflect those results in the prose.

Frequency and intensity, p. 21

Duration of the experience, p. 23

It's at least as great a pity that they didn't present their results tabularly either.

What's the sample size? Where are the confidence intervals?

But the pictures are pretty, eh?

RfB

[Vigilant]

* Whether wrestling (a male-only sport) should be continued in the Olympics.

Point of order: Olympic wrestling is not a male-only sport.

P.S.
https://en.wikipedia.org/wiki/Wrestling ... pics#Women

That wrestling article is a total mess.

[Earthy Astringent]

* Whether wrestling (a male-only sport) should be continued in the Olympics.

Point of order: Olympic wrestling is not a male-only sport.

I don't know if I'm pleased or sad at learning that "Jello Wrestling" is a red link.

As to the survey itself, "amateur hour" is in apt description. Mind you, I'm not a professional in market research but I have the humidity to admit I know what I don't know, which is more to say than clown posse who put this "report" together. Much has been said about the "revenge porn" question. How that question made it pass multiple reviewers before it got published is beyond me.

With the size of their endowment, they could have paid for a real survey to be assembled, though in all likelihood they would have picked some political stooge with their own agenda. I swear to God, the WMF reminds me of Dicken's Miss Havisham. Rich, petty, and insane.

[thekohser]

I swear to God, the WMF reminds me of Dicken's Miss Havisham. Rich, petty, and insane.

I wouldn't have pegged you as a reader of Dicken.

[Earthy Astringent]

I swear to God, the WMF reminds me of Dicken's Miss Havisham. Rich, petty, and insane.

I wouldn't have pegged you as a reader of Dicken.

I didn't know you were into pegging.

[Zoloft]

Do not derail this discussion with trivial corrections of punctuation and sexually-themed jokes.

[eagle]

* Whether wrestling (a male-only sport) should be continued in the Olympics.

Point of order: Olympic wrestling is not a male-only sport.

P.S.
https://en.wikipedia.org/wiki/Wrestling ... pics#Women

That wrestling article is a total mess.

I am very sorry that I missed that important development. Most of the Olympic sports that only allowed male athletes to compete including Boxing and Wrestling now have slots for women to compete as well. Indeed, following the inclusion of Women's Basketball in the Olympics in 1976, I do not understand the argument that Netball must also be included as a matter of gender equity. If feminists believe that women athletes should compete at the Olympic level, let women athletes take jump shots, dribble, and go for 3-point shots on the same terms as male athletes. The general public expects a certain level of athleticism in order to maintain spectator interest, and more people are attracted to watch basketball rather than netball.

The bottom line is that WP should be able to have an honest, factual discussion of gender-related Olympic issues without throwing down the "harassment card".

[Randy from Boise]

Do not derail this discussion with trivial corrections of punctuation and sexually-themed jokes.

That actually sounds like a great thread title...

RfB

[Johnny Au]

Don't forget about baseball vs. softball and how when the Olympics had those sports, only men could play baseball and women played softball.

[Earthy Astringent]

Don't forget about baseball vs. softball and how when the Olympics had those sports, only men could play baseball and women played softball.

Those were excluded solely because of the anti-American/pro-Europe bias that permeates the IOC. While baseball perceived as a US sport (with good reason) it is extremely popular in Latin and Southern America and some Asian countries. The IOC's level of corruption rivals that of FIFA.

While some want to make the sport selection a gender based controversy, IMO it has more to do with getting events that specific countries are expected to dominate.

[Anroth]

A friend of mine worked on some Olympic branded videogames, he described the IOC as the most corrupt organisation in existence.

[Poetlister]

Don't forget about baseball vs. softball and how when the Olympics had those sports, only men could play baseball and women played softball.

Those were excluded solely because of the anti-American/pro-Europe bias that permeates the IOC. While baseball perceived as a US sport (with good reason) it is extremely popular in Latin and Southern America and some Asian countries. The IOC's level of corruption rivals that of FIFA.

While some want to make the sport selection a gender based controversy, IMO it has more to do with getting events that specific countries are expected to dominate.

That's why cricket isn't an Olympic sport, despite its popularity in India, Pakistan and the English-speaking Caribbean.

[GorillaWarfare]

The 3% "other gender" includes people who may not accurately state their gender.

If people are inaccurately stating their gender, wouldn't this also affect the male and female groups? This group is simply for people who do not fall within the binary male/female options.

[thekohser]

Don't forget about baseball vs. softball and how when the Olympics had those sports, only men could play baseball and women played softball.

Those were excluded solely because of the anti-American/pro-Europe bias that permeates the IOC. While baseball perceived as a US sport (with good reason) it is extremely popular in Latin and Southern America and some Asian countries. The IOC's level of corruption rivals that of FIFA.

While some want to make the sport selection a gender based controversy, IMO it has more to do with getting events that specific countries are expected to dominate.

That's why cricket isn't an Olympic sport, despite its popularity in India, Pakistan and the English-speaking Caribbean.

You guys have successfully derailed this thread.

[eagle]

The 3% "other gender" includes people who may not accurately state their gender.

If people are inaccurately stating their gender, wouldn't this also affect the male and female groups? This group is simply for people who do not fall within the binary male/female options.

I did not mean to trigger a debate. What I was trying to express is that it is possible that transgendered people will select "male" or "female" based on their current gender, but that pranksters will select "other genders." So if you have 15 to 30 people in the "other gender" group a further breakdown of that group will not be reliable. It will not tell you whether transgendered people have experienced harassment. It will not tell you anything about people who refused to state their gender because of concerns with the format of the question have experienced harassment because of the status that underpins their concerns.

The survey was not well designed.

[Drijfzand]

I'm surprised that people (WP, WMF..) accept the validity of the results (or is there a discussion on WP that I've missed?). It seems obvious that the data contains hundreds of fake harassment reports claiming to have experienced every item on the list.
How likely is it that the lower half of the list (doxxing, hacking, revenge porn, impersonation and threats of violence) fall within a narrow interval of 64±3?
Or, if we assume that those five always appear in the "Yes" group, never in the "Unsure" group, we find that of all the people who reported being harassed, only 14.5% where not doxxed, 16.3% did not receive threats of violence. 19.8% were not hacked. 16.8% were not impersonated. 22% were not a victim of revenge porn.
(Another representation of the same data, not sure if it adds anything.)

I cannot prove that the survey contains fake or flawed harassment reports, but I can show what the results look like without them. corresponding frequency data (sum of Often and Occasionally) from figure 35 "forms of harassment witnessed and how frequently they are witnessed" was added at the bottom, for comparison purposes.

Name calling 26+31=57
Trolling 31+30=61
Content Vandalism 35+32=67
Outing/Doxxing 5+15=20
Hacking 1+5=6
Impersonation 7+17=24
Revenge porn 1+2=3
Stalking 12+22=34
Threats of violence 4+11=15
Discrimination 17+25=42

Original data assuming 50 - 100 - ... - 729 bad reports.

Code: Select all

        vandal  trol  namec  discr  stalk   out   threat  imper  hack  revpo
729:     63     58     48     35     35     18     13     13      8      3
700:     64     60     50     38     38     21     17     17     12      7
650:     68     63     55     44     44     29     25     25     20     16
600:     70     66     58     49     49     35     31     31     27     23
550:     73     69     62     52     52     40     36     36     32     29
500:     74     71     64     56     56     44     40     40     37     34
450:     76     73     67     59     59     48     44     44     41     38
400:     78     75     69     61     61     51     48     48     45     42
350:     79     76     70     63     63     54     51     51     48     45
300:     80     77     72     65     65     56     53     53     51     48
250:     81     79     74     67     67     58     56     56     53     51
150:     83     81     76     70     70     62     60     60     58     56
100:     84     81     77     72     72     64     62     62     60     57
50:      84     82     78     73     73     66     63     63     61     59

witn:    67     61     57     42     34     20     15     24      6      3

729 fake/faulty reports seems about right, but the witnessed data isn't necessarily an accurate measure.
Anyway, the main reasons why the data must be wrong are:

Harassment types experienced: 1215 responses.
five least common forms of harassment:
- 61% (740) revenge porn;
- 63% (760) hacking;
- 65% (795) threats of violence;
- 65% (789) impersonation,
- 67% (810) doxxing.

- hacking: there are lots of ways to harass someone, but it's not like you can simply decide to hack his account. If an editors account (on or off WP) is hacked, it will be a crime of opportunity, or one of a group of targets. The idea that in most cases of severe harassment a person's account would be hacked is ridiculous. 1215 cases of harassment or possible harassment reported, and account hacked in 760 of them?!
- revenge porn and doxxing: same reason.
- doxxing, revenge porn, hacking, impersonation, threats of violence: all within 775±5% Simply can't happen.

[Randy from Boise]

I'm surprised that people (WP, WMF..) accept the validity of the results (or is there a discussion on WP that I've missed?). It seems obvious that the data contains hundreds of fake harassment reports claiming to have experienced every item on the list.

How likely is it that the lower half of the list (doxxing, hacking, revenge porn, impersonation and threats of violence) fall within a narrow interval of 64±3?
Or, if we assume that those five always appear in the "Yes" group, never in the "Unsure" group, we find that of all the people who reported being harassed, only 14.5% where not doxxed, 16.3% did not receive threats of violence. 19.8% were not hacked. 16.8% were not impersonated. 22% were not a victim of revenge porn.
(Another representation of the same data, not sure if it adds anything.)

I cannot prove that the survey contains fake or flawed harassment reports, but I can show what the results look like without them. corresponding frequency data (sum of Often and Occasionally) from figure 35 "forms of harassment witnessed and how frequently they are witnessed" was added at the bottom, for comparison purposes.

Name calling 26+31=57
Trolling 31+30=61
Content Vandalism 35+32=67
Outing/Doxxing 5+15=20
Hacking 1+5=6
Impersonation 7+17=24
Revenge porn 1+2=3
Stalking 12+22=34
Threats of violence 4+11=15
Discrimination 17+25=42

Original data assuming 50 - 100 - ... - 729 bad reports.

Code: Select all

        vandal  trol  namec  discr  stalk   out   threat  imper  hack  revpo
729:     63     58     48     35     35     18     13     13      8      3
700:     64     60     50     38     38     21     17     17     12      7
650:     68     63     55     44     44     29     25     25     20     16
600:     70     66     58     49     49     35     31     31     27     23
550:     73     69     62     52     52     40     36     36     32     29
500:     74     71     64     56     56     44     40     40     37     34
450:     76     73     67     59     59     48     44     44     41     38
400:     78     75     69     61     61     51     48     48     45     42
350:     79     76     70     63     63     54     51     51     48     45
300:     80     77     72     65     65     56     53     53     51     48
250:     81     79     74     67     67     58     56     56     53     51
150:     83     81     76     70     70     62     60     60     58     56
100:     84     81     77     72     72     64     62     62     60     57
50:      84     82     78     73     73     66     63     63     61     59

witn:    67     61     57     42     34     20     15     24      6      3

729 fake/faulty reports seems about right, but the witnessed data isn't necessarily an accurate measure.
Anyway, the main reasons why the data must be wrong are:

Harassment types experienced: 1215 responses.
five least common forms of harassment:

- 61% (740) revenge porn;
- 63% (760) hacking;
- 65% (795) threats of violence;
- 65% (789) impersonation,
- 67% (810) doxxing.

- hacking: there are lots of ways to harass someone, but it's not like you can simply decide to hack his account. If an editors account (on or off WP) is hacked, it will be a crime of opportunity, or one of a group of targets. The idea that in most cases of severe harassment a person's account would be hacked is ridiculous. 1215 cases of harassment or possible harassment reported, and account hacked in 760 of them?!
- revenge porn and doxxing: same reason.
- doxxing, revenge porn, hacking, impersonation, threats of violence: all within 775±5% Simply can't happen.

The primary structural problem is that this was a survey for which its creators sent out emails and posted to mailing lists, "hey, come take our survey." A serious survey to gauge the incidence and severity of various forms of harassment would have made use of random participants rather than opting for a wide-open, unfiltered approach.

A survey of self-identified harassment victims would also be valuable, but that would have altogether different inclusion parameters and it wouldn't be "marketed" as a survey of harassment levels of Wikipedians in general, but rather one investigating the forms and outcomes of a finite group of incidents taking place to a subset of Wikipedians.

The old mantra of Garbage-In-Garbage-Out seems to be applicable to the current report, pretty pictures and graphs notwithstanding.

RfB

[Randy from Boise]

Well, THIS is interesting...

linkhttps://meta.wikimedia.org/wiki/Researc ... urvey_2015[/link]

Invitation methodology

The survey could be accessed through a central notice shown to a percentage of every project's users, as well as through direct invitation to a selection of dormant contributors. The central notice was not visible through all projects at the same time. It was rolled out in phases, starting with the smaller wikis first and gradually working its way towards the larger projects too, culminating in the English Wikipedia. The phased roll-out was intended to allow users from smaller projects the opportunity to participate in the survey before the response limit was hit.

I know I took the survey and my distinct recollection was that I responded to a link to it off of GenderGap-l.

This indicates that they actually DID attempt to randomize participation, which means something else is going on with the data.

And here are the questions asked...

https://meta.wikimedia.org/wiki/Researc ... /Questions

Part I : Your own experience[edit]
In this survey, harassment may refer to on-wiki or off-wiki incidents, where the off-wiki incident relates to your participation in Wikimedia projects. Editing may refer to contributing content to the projects in any form, including uploading images or performing administrative tasks.
1. How often do you currently edit in the Wikimedia projects?
More than once per day
Once or twice per week
Once or twice per month
Less than once per month
I no longer edit or contribute content in the Wikimedia projects
[IF #1 = "I no longer edit or contribute content in the Wikimedia projects" IS SELECTED, SKIP TO #5 = Have you experienced harassment in the Wikimedia projects?]
2. How often do you engage with other users online (on-wiki or off-wiki) in the following ways?
[ACROSS]
Once a day or more
Once or twice per week
Once or twice per month
Less than once a month
Never
[DOWN]
Collaborating with other users on a project or topic
Having friendly conversations with other users
Giving or receiving direct feedback about your work
Having arguments with other users
Other (please specify)

3. To what extent are you satisfied or dissatisfied with the following ways of engaging socially with other users?
[ACROSS]
Very satisfied
Satisfied
Neither satisfied nor dissatisfied
Dissatisfied
Very dissatisfied
Not applicable/Not sure
[DOWN]
Collaborating with other users on a project or topic
Having friendly conversations with other users
Giving or receiving direct feedback about your work
Having arguments with other users
Other (please specify)

4. To what extent do you agree or disagree with the following statements about working on Wikimedia projects?
[ACROSS]
Strongly agree
Agree
Neither disagree nor agree
Disagree
Strongly disagree
[DOWN]
I enjoy collaborating with other users
I receive useful feedback about my contributions
I find other users supportive and helpful
Other users seem to enjoy collaborating with me
I try to give useful feedback to other users
I try to be constructive on talk pages
Other users find my support helpful

5. Have you experienced harassment in the Wikimedia projects?
Yes
No
I am not sure
[IF #5 = "No" IS SELECTED, SKIP TO #12 = To what extent are you aware of the following resolution paths within the Wikimedia projects?]
6. How many times have you experienced incidents like the ones described below while working on any of the Wikimedia projects?
Please remember that filling in this survey is not an alternative to reporting abuse and will not generate a response. If you need to report an incident, please use existing reporting methods.
Drag the slider to select the number of times you experienced each type of harassment. 0 can be entered by clicking on the slider once. For values of 100+, select 100.
[ACROSS]
0 10 20 30 40 50 60 70 80 90 100
[DOWN]
I have been called offensive names [Name calling]
I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming]
My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism]
My personal information has been published without my consent [Outing / Doxxing]
Access to my private accounts has been compromised [Hacking]
Another individual pretended to be me [Impersonation]
Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn]
I have received unwanted and obsessive attention that made me feel scared [Stalking]
I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence]
I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination]
Other experience (please specify)

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
7. Which of the following statements describes your experience of harassment?
It was done by one person through a single episode
It was done by one person through multiple episodes
It was done by multiple people through a single episode
It was done by multiple people through multiple episodes

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
8. How long did the harassment last? (overall duration)
Less than one hour
Less than one day
More than one day but less than one week
More than one week but less than one month
More than one month but less than one year
One year or more

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
9. On which project(s) did you experience this harassment?
Wikipedia
Wiktionary
Wikiquote
Wikibooks
Wikisource
Wikinews
Wikiversity
Wikispecies
MediaWiki
Wikidata
Wikimedia Commons
Wikivoyage
Wikimedia Meta-Wiki
Wikimedia Incubator
Wikimedia Labs
Wikimedia Foundation
Prefer not to say
Off-wiki (please specify)

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
10. What was the harassment based on?
Age
Appearance
Ethnicity
Disability
Gender identity
Political ideas
Race
Religion
Sexual orientation
Don't know/Not sure
Other (please specify)

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
11. If you'd like, share an example/quote of the harassment text that you have received, or you may skip this question. You can quote short excerpts of the harassing text that you have received, captions of photos, etc. If you were harassed through imagery rather than text, you are welcome to state that.
I prefer to skip this question
Please specify

12. To what extent are you aware of the following resolution paths within the Wikimedia projects?
Consulting with another community member
Opening a discussion on the Village Pump / Community discussion board
Escalating to an Arbitration Committee
Opening a discussion on Meta
Contacting the Community Advocacy Team at the Wikimedia Foundation
Other (please specify)

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
13. In which of the following ways have you responded to harassment that you experienced?
I did not react / Ignored the incident
I asked the user to stop the harassing behavior
I discussed with other community members
I discussed with friends / family
I consulted a professional for emotional support (e.g. therapist)
I consulted legal counsel / took legal action
I reported the incident(s) through the Wikimedia project
I contacted law enforcement
I contacted the Wikimedia Foundation for help
Other (please specify)

[ASK ONLY IF #13 = I contacted the Wikimedia Foundation for help]
14. How satisfied were you with the support that you received from the Foundation?
Very satisfied
Satisfied
Neither satisfied nor dissatisfied
Dissatisfied
Very dissatisfied
Prefer not to say
Other (please specify)

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
15. To what extent were your actions effective in resolving the problem?
[ACROSS]
Not at all
A little
Somewhat
Mostly
Completely
Not Applicable
[DOWN]
I did not react / Ignored the incident
I asked the user to stop the harassing behavior
I discussed with other community members
I discussed with friends / family
I consulted a professional for emotional support (e.g. therapist)
I consulted legal counsel / took legal action
I reported the incident(s) through the Wikimedia project
I contacted law enforcement
I contacted the Wikimedia Foundation for help
Other (please specify)

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
16. To what extent did you find your experience(s) of harassment in the Wikimedia projects distressful?
Extremely upsetting
Very upsetting
Somewhat upsetting
A little upsetting
Not at all upsetting

[ASK ONLY if #6 = I have been called offensive names [Name calling] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received deliberately nasty messages aimed at upsetting me or provoking an angry response [Trolling / Flaming] IS GREATER THAN OR EQUAL TO 1
OR #6 = My user page has been vandalized or my article contributions have been unfairly challenged, devalued or deleted [Content vandalism] IS GREATER THAN OR EQUAL TO 1
OR #6 = My personal information has been published without my consent [Outing / Doxxing] IS GREATER THAN OR EQUAL TO 1
OR #6 = Access to my private accounts has been compromised [Hacking] IS GREATER THAN OR EQUAL TO 1
OR #6 = Another individual pretended to be me [Impersonation] IS GREATER THAN OR EQUAL TO 1
OR #6 = Sexually explicit or sexualised photos of me have been published without my consent [Revenge porn] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received unwanted and obsessive attention that made me feel scared [Stalking] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have received threats of violence, sexual or otherwise, against myself or another person close to me (family member, spouse, etc.) [Threats of violence] IS GREATER THAN OR EQUAL TO 1
OR #6 = I have been treated differently/unfairly based on personal characteristics, instead of merit [Discrimination] IS GREATER THAN OR EQUAL TO 1
OR #6 = Other experience (please specify) IS GREATER THAN OR EQUAL TO 1]
17. To what extent has your personal experience of harassment affected your involvement in the following activities?
[ACROSS]
Greatly decreased
Decreased
Neither increased nor decreased
Increased
Greatly increased
Not sure/Don't know
[DOWN]
Participating in Wikimedia projects online
Participating in a Wikimedia project in which I experienced harassment
Participating in a new Wikimedia project in which I did not experience harassment
Interacting with other users online

Part II : The experience of other community members[edit]
This section focuses on the direct and indirect effects of harassment caused by witnessing other community members being harassed.

18. Have you witnessed others experiencing harassment in the Wikimedia projects?
Yes
No
I am not sure
[IF #18 = "No" IS SELECTED, SKIP TO #24 = What ideas or improvements would you like to see implemented in the movement’s approach to harassment?]
19. How often have you witnessed others experiencing the following forms of harassment while working on any of the Wikimedia projects?
[ACROSS]
Often
Occasionally
Rarely
Never
[DOWN]
Being called offensive names [Name calling]
Receiving deliberately nasty messages aimed at upsetting or provoking an angry response [Trolling / Flaming]
User page being vandalized or article contributions being unfairly challenged, devalued or deleted [Content vandalism]
Personal information has been published without consent [Outing / Doxxing]
Access to another person's private accounts has been compromised [Hacking]
A individual pretending to be somebody else [Impersonation]
Sexually explicit or sexualised photos of another community member being published without consent [Revenge porn]
Unwanted and obsessive attention that made the other person feel scared [Stalking]
Threats of violence, sexual or otherwise, against the community member or another person close to them (family member, spouse, etc.) [Threats of violence]
Being treated differently/unfairly based on personal characteristics, instead of merit [Discrimination]
Other experience (please specify)

20. In what way, if any, did you get involved with the harassment that you witnessed?
I confronted user who was harassing
I offered support to the user who was being harassed
I reached out to the user who was being harassed to get more details so as to build a stronger, joint case for our experience
I attempted to mediate
I did not react
Other (please specify)

21. To what extent were your actions effective to resolve harassment that you witnessed ?
Not at all
A little
Somewhat
Mostly
Completely

22. How often did you get harassed yourself as a result of being involved in the situation?
Always
Often
Sometimes
Rarely
Never

23. How did the witnessing of another community member being harassed affect your own participation to the Wikimedia projects?
I considered not contributing any more
I stopped contributing for a while, but ultimately I continued contributing
I stopped contributing overall
It didn't’t change my involvement
I started contributing more

24. What ideas or improvements would you like to see implemented in the movement’s approach to harassment?
We welcome feedback on various aspects of the process, from the reporting mechanisms available to you, ease of finding them, to improving the support you received.
Part III : Your profile[edit]
This section is optional and will only require 2 minutes of your time.
25. What is your current gender identity?
Male
Female
Trans male / Trans man
Trans female / Trans woman
Gender queer / Gender non-conforming
Prefer not to say
Different identity (please specify)

26. What is your age?
17 or under
18-24
25-34
35-44
45-54
55-64
65 or more
Prefer not to say

27. To what extent do you consider yourself to be racially or culturally different from the majority of editors, within the Wikimedia project you are most active in?
Extremely different
Very different
Somewhat different
A little different
Not at all different
Prefer not to say

28. To what extent have you shared with other users the following information in any way on Wikimedia projects?
[ACROSS]
Always
Usually
Sometimes
Rarely
Never
Prefer not to say
[DOWN]
Gender
Sexual Orientation
Age
Ethnic or racial group
Political ideas
Location
Religious beliefs

29. What is the highest degree or level of education you have completed?
Less than high school
High school graduate or equivalent
Some college, no degree
Associate’s degree
Bachelor’s degree
Graduate or professional degree (Masters, etc.)
Ph.D.
Other
Prefer not to say

30. How long have you actively participated in the Wikimedia projects? (years)
Less than 1 year
2
3
4
5
6
7
8
9
10
11
12
13
14 or more
Prefer not to say

31. Which Wikimedia project do you participate in the most?
Wikipedia
Wiktionary
Wikiquote
Wikibooks
Wikisource
Wikinews
Wikiversity
Wikispecies
MediaWiki
Wikidata
Wikimedia Commons
Wikivoyage
Wikimedia Meta-Wiki
Wikimedia Incubator
Wikimedia Labs
Wikimedia Foundation
Prefer not to say

32. Which language do you mostly edit in?
العربية
English
Deutsch
Español
Français
贛語
Italiano
日本語
Nederlands
Polski
Português
Русский
Prefer not to say
Other (please specify)

RfB

[Vigilant]

If the data from this survey is even remotely correct, why does the WMF still allow anonymous editing?

[Jim]

Well, THIS is interesting...

linkhttps://meta.wikimedia.org/wiki/Researc ... urvey_2015[/link]

I know I took the survey and my distinct recollection was that I responded to a link to it off of GenderGap-l.

I took it too, and I can confirm it was via a site-notice, so I must have been one of the "percentage of every project's users" it was shown to.

However, I can also confirm it was extremely poorly designed, difficult to answer, and as someone who does not consider they have been harassed, I felt it was almost begging me to say I had, or give details of 3rd party harassment I'd witnessed.

In some ways that's understandable if you're trying to find out what kinds of harassment exist (although it didn't do that at all well, either), but as a measure of prevalence? Utterly useless and malformed.

One of the big issues is that wikipedia already uses the term "harassment" in such a wide, loose, wacky undefined way that the foundation of the survey is unsound without a better definition, and the result is that anyone who once didn't get their own way in an edit war about anything may easily frame that as "harassment", and the survey, as presented, tends to encourage that.

[Randy from Boise]

Assuming that this was indeed a randomized survey (and it seems that it was, my memory notwithstanding) it will have interesting things to say about the demographics of Wikipedia. Back to that later.

The problematic page is actually p. 17, which is said to contain the percentage of different types of harassment said to have been experienced by those who have been harassed. As mentioned above, the quantity listed for "revenge porn" fails the test of common sense: 61% of the 35% of 2,495 respondents = 533 people (21.35%), which seems patently absurd for a random group of mostly male, mostly anonymous participants on a web site.

There is probably a software defect at play here, my guess. There is clearly something awry, as mentioned by others above in this thread.

==========

Now as for demographics, of those responding M or F (rather than other or declining to respond), the survey indicates a M:F ratio of 88:12. There's the first good measure of the gender gap that we have had for a couple years. "Prefer not to say" is 7% of the total, so even if that was 100% female, which it was not, that would indicate a top end estimate of 20% women (11+9%). (pg. 11).

Finally, some decent age data. As anticipated, the age of Wikipedians is older than the stereotypical young 20-something.. Fully 35% of survey respondents are age 45 or older, with the college age or younger set (<25 years old) a mere 24% of respondents. Median age would seem to be in the low 30s. (So much for the 16-year olds playing a video game meme...) (pg. 12)

RfB

[Randy from Boise]

Having spent some more time this morning with the document and its background methodology, I'm now ready to retract my use of "propaganda" in connection with it.

The design is weak, the presentation lame, and the main set of harassment data seems to have some sort of major, fundamental, systemic defect.

But it's not "propaganda."

RfB

[Jim]

Finally, some decent age data. As anticipated, the age of Wikipedians is older than the stereotypical young 20-something.. Fully 35% of survey respondents are age 45 or older, with the college age or younger set (<25 years old) a mere 24% of respondents. Median age would seem to be in the low 30s. (So much for the 16-year olds playing a video game meme...) (pg. 12)

But isn't that just an example of how bad statistics can "prove" anything you want them to?
Why can't it just be interpreted as showing that wikipedia editors over 45 are more likely to respond to crappy surveys like this, for instance? I am, and I did.

[snowskarlet]

Well, THIS is interesting...

linkhttps://meta.wikimedia.org/wiki/Researc ... urvey_2015[/link]

I know I took the survey and my distinct recollection was that I responded to a link to it off of GenderGap-l.

I took it too, and I can confirm it was via a site-notice, so I must have been one of the "percentage of every project's users" it was shown to.

I too was a random pick, since I just happened to see a site-notice on the particular day I took the survey. I know that for sure, because I only haunt English Wiki and Simple Wiki. (I wonder now which it was?)

[Randy from Boise]

Finally, some decent age data. As anticipated, the age of Wikipedians is older than the stereotypical young 20-something.. Fully 35% of survey respondents are age 45 or older, with the college age or younger set (<25 years old) a mere 24% of respondents. Median age would seem to be in the low 30s. (So much for the 16-year olds playing a video game meme...) (pg. 12)

But isn't that just an example of how bad statistics can "prove" anything you want them to?
Why can't it just be interpreted as showing that wikipedia editors over 45 are more likely to respond to crappy surveys like this, for instance? I am, and I did.

Similarly, it might be argued that victims of harassment are more apt to respond to a survey like this than non-victims. That's another design flaw, it shouldn't have been "promoted" as a survey on harassment, which I recall that it was.

But on age: it's what we've got — which is a Trumpload more than WMF has ever before provided us. Definitely one good set of data points on age, we need more — but this very much illuminates what I've been chanting for several years now. Wikipedians are not a bunch of college kids, the young fresh faces of Wikimania gatherings (many there via travel grants) notwithstanding. It's an older crowd than that, especially the content people.

RfB

[snowskarlet]

...and I specifically identified as female, since my main sorrow I mentioned was that I was:

“hurtfully mocked for my gender and an illness,”

..and I truthfully ticked my age grouping (and I too am beyond anything know as youthful ( except in spirit )

[Jim]

Similarly, it might be argued that victims of harassment are more apt to respond to a survey like this than non-victims.

I would strongly argue that too. Certainly they might be more inclined to plod all the way through to the end of that mess of a thing.
I only did so because I wanted to see all the questions, otherwise I would have given up after 2 pages.

[Randy from Boise]

One thing that bears mentioning is that this survey somewhat overrepresents English-Wikipedia. About 45% of respondents came from En-WP, compared to something around a 35% share of Very Active Wikipedians each month.

RfB

[Jim]

( except in spirit )

As if anything else matters. I'm glad I missed that memo, if I did.

[Jim]

One thing that bears mentioning is that this survey somewhat overrepresents English-Wikipedia. About 45% of respondents came from En-WP, compared to something around a 35% share of Very Active Wikipedians each month.

You're right, but there comes a point where it's just better to say "Ok. The targeting was shit, the questions were shit, the analysis was shit - we're just not any good at this", and get someone grown-up to do it.

[Kingsindian]

I'm surprised that people (WP, WMF..) accept the validity of the results (or is there a discussion on WP that I've missed?). It seems obvious that the data contains hundreds of fake harassment reports claiming to have experienced every item on the list.

How likely is it that the lower half of the list (doxxing, hacking, revenge porn, impersonation and threats of violence) fall within a narrow interval of 64±3?
Or, if we assume that those five always appear in the "Yes" group, never in the "Unsure" group, we find that of all the people who reported being harassed, only 14.5% where not doxxed, 16.3% did not receive threats of violence. 19.8% were not hacked. 16.8% were not impersonated. 22% were not a victim of revenge porn.
(Another representation of the same data, not sure if it adds anything.)

I cannot prove that the survey contains fake or flawed harassment reports, but I can show what the results look like without them. corresponding frequency data (sum of Often and Occasionally) from figure 35 "forms of harassment witnessed and how frequently they are witnessed" was added at the bottom, for comparison purposes.

Name calling 26+31=57
Trolling 31+30=61
Content Vandalism 35+32=67
Outing/Doxxing 5+15=20
Hacking 1+5=6
Impersonation 7+17=24
Revenge porn 1+2=3
Stalking 12+22=34
Threats of violence 4+11=15
Discrimination 17+25=42

Original data assuming 50 - 100 - ... - 729 bad reports.

Code: Select all

        vandal  trol  namec  discr  stalk   out   threat  imper  hack  revpo
729:     63     58     48     35     35     18     13     13      8      3
700:     64     60     50     38     38     21     17     17     12      7
650:     68     63     55     44     44     29     25     25     20     16
600:     70     66     58     49     49     35     31     31     27     23
550:     73     69     62     52     52     40     36     36     32     29
500:     74     71     64     56     56     44     40     40     37     34
450:     76     73     67     59     59     48     44     44     41     38
400:     78     75     69     61     61     51     48     48     45     42
350:     79     76     70     63     63     54     51     51     48     45
300:     80     77     72     65     65     56     53     53     51     48
250:     81     79     74     67     67     58     56     56     53     51
150:     83     81     76     70     70     62     60     60     58     56
100:     84     81     77     72     72     64     62     62     60     57
50:      84     82     78     73     73     66     63     63     61     59

witn:    67     61     57     42     34     20     15     24      6      3

729 fake/faulty reports seems about right, but the witnessed data isn't necessarily an accurate measure.
Anyway, the main reasons why the data must be wrong are:

Harassment types experienced: 1215 responses.
five least common forms of harassment:

- 61% (740) revenge porn;
- 63% (760) hacking;
- 65% (795) threats of violence;
- 65% (789) impersonation,
- 67% (810) doxxing.

- hacking: there are lots of ways to harass someone, but it's not like you can simply decide to hack his account. If an editors account (on or off WP) is hacked, it will be a crime of opportunity, or one of a group of targets. The idea that in most cases of severe harassment a person's account would be hacked is ridiculous. 1215 cases of harassment or possible harassment reported, and account hacked in 760 of them?!
- revenge porn and doxxing: same reason.
- doxxing, revenge porn, hacking, impersonation, threats of violence: all within 775±5% Simply can't happen.

The primary structural problem is that this was a survey for which its creators sent out emails and posted to mailing lists, "hey, come take our survey." A serious survey to gauge the incidence and severity of various forms of harassment would have made use of random participants rather than opting for a wide-open, unfiltered approach.

A survey of self-identified harassment victims would also be valuable, but that would have altogether different inclusion parameters and it wouldn't be "marketed" as a survey of harassment levels of Wikipedians in general, but rather one investigating the forms and outcomes of a finite group of incidents taking place to a subset of Wikipedians.

The old mantra of Garbage-In-Garbage-Out seems to be applicable to the current report, pretty pictures and graphs notwithstanding.

RfB

Look at how a serious survey organization like Pew tries to ensure that the sample is representative. Also present: sample sizes and error rates.

[Drijfzand]

Here's your problem!

The page with sliders on which forms of harassment I have experienced would not advance until they were all non-zero. I selected a distinct number and added a note to say that actually meant zero. Burninthruthesky (talk) 16:39, 3 November 2015 (UTC)

https://meta.wikimedia.org/wiki/Researc ... sign_fault

It was launched on 2 November. May have been 36 to 48 hours online before fixed.

The tech issue on the multi-statement questions (like the one with the sliders) should be now fixed and no longer forcing participants to enter a number in order to progress. You will still get a reminder that "you have not answered all questions/statements" if some of the statements are unanswered but it should no longer prevent you from moving on to the next question. Do keep in mind that it is still not possible to move backwards into the survey. So, if you chose to leave statements unanswered, you won't be able to return to them later. Thank you all for raising your concerns as this did not appear to be an issue during the testing phase, before the survey's launch. If you become aware of other such glitches, do let us know.Kalliope (WMF) (talk) 09:15, 4 November 2015 (UTC)

Yet no one seems to have done anything about the responses they received in that period. And no one noticed something was wrong with the data when they analyzed it and wrote the report.
How obvious does it have to be before they make the connection???

[Randy from Boise]

Here's your problem!

The page with sliders on which forms of harassment I have experienced would not advance until they were all non-zero. I selected a distinct number and added a note to say that actually meant zero. Burninthruthesky (talk) 16:39, 3 November 2015 (UTC)

https://meta.wikimedia.org/wiki/Researc ... sign_fault

It was launched on 2 November. May have been 36 to 48 hours online before fixed.

The tech issue on the multi-statement questions (like the one with the sliders) should be now fixed and no longer forcing participants to enter a number in order to progress. You will still get a reminder that "you have not answered all questions/statements" if some of the statements are unanswered but it should no longer prevent you from moving on to the next question. Do keep in mind that it is still not possible to move backwards into the survey. So, if you chose to leave statements unanswered, you won't be able to return to them later. Thank you all for raising your concerns as this did not appear to be an issue during the testing phase, before the survey's launch. If you become aware of other such glitches, do let us know.Kalliope (WMF) (talk) 09:15, 4 November 2015 (UTC)

Yet no one seems to have done anything about the responses they received in that period. And no one noticed something was wrong with the data when they analyzed it and wrote the report.
How obvious does it have to be before they make the connection???

Bing! Bing! Bing! Bing! — We have a winner!

This study is pure GIGO as a quantification of harassment, but it actually gives us an excellent snapshot of the gender gap and the age of Wikipedia participants, so all is not lost.

RfB

[Jim]

it actually gives us an excellent snapshot of the gender gap and the age of Wikipedia participants

{{cn}}

[Randy from Boise]

Here's your problem!

The page with sliders on which forms of harassment I have experienced would not advance until they were all non-zero. I selected a distinct number and added a note to say that actually meant zero. Burninthruthesky (talk) 16:39, 3 November 2015 (UTC)

https://meta.wikimedia.org/wiki/Researc ... sign_fault

It was launched on 2 November. May have been 36 to 48 hours online before fixed.

Actually, the glitch and the bad data it generated lasted longer than that.

To maximize participation from all projects and minimize error, the survey was released gradually over the course of the first five days based on project size, starting with release to the smaller communities before its opening to the larger projects. (pg. 2)

So the defective sliders that would not accept a quantity of 0 were up for as many as 4 days on other projects before going up at En-WP, with more bad data generated until the fix.

What we have here is a failure to adequately beta test the software.

Sound familiar???

RfB

[SneakySasha]

anyone who once didn't get their own way in an edit war about anything may easily frame that as "harassment", and the survey, as presented, tends to encourage that.

This is the very argument used to silence those editors who believe they are being harassed. And oftentimes those editors were actually harassed in an edit war by a bunch of POV-pushing editors, so while there is some connection, it really is no longer about "didn't get their own way in an edit war," it's now about all the harassment that was heaped upon them so those POV pushers can get their way!

[Jim]

So the defective sliders that would not accept a quantity of 0 were up for as many as 4 days on other projects before going up at En-WP, with more bad data generated until the fix.What we have here is a failure to adequately beta test the software.

They certainly made my vote hard, because zero was my option for lots of the questions, and I do distinctly recollect it being hard to express that. The stupid sliders were certainly a factor. I found them very frustrating,

[thekohser]

This indicates that they actually DID attempt to randomize participation, which means something else is going on with the data.

What did the invitation say?

It's one thing to say you're conducting a survey across a probability sample, please participate.

It's another thing to say you're conducting a survey about the harassment we experience on Wikimedia projects, please participate.

[Jim]

What did the invitation say?
It's one thing to say you're conducting a survey across a probability sample, please participate.
It's another thing to say you're conducting a survey about the harassment we experience on Wikimedia projects, please participate.

It said the latter, from memory. I agree.

[Kelly Martin]

On page 17, it says 1,215 people responded to a 'Forms of harassment' question, with 85% reporting content vandalism as their most frequent form of harassment. 63% are claiming they've been hacked and 61% are claiming they have been targeted for revenge porn, which seems improbable, or I would hope so and there really haven't been hundreds of wikipediots getting hacked and having revenge porn of them uploaded.

What this tells me is that some Wikipedians used a nonstandard definition of "harassment" when responding to the survey, and that the researchers did not conduct interviews of respondents to determine whether respondents were using the term "harassment" to mean what normal people think it means, or if they instead had some idiosyncratic meaning for the term.

It would appear that this study is worthless, insofar as it appears to rely primarily on unguided self-selected survey responses. At most all you can use this for is to get a sense of the possible landscape; you certainly cannot draw any quantitative conclusions from this nearly useless dataset.

[SneakySasha]

I'm wondering if any of you have ever actually spent any time on any forums where the majority of the users are women? 'Cause overall, it's much, much less of what I can only classify as a "war-like" environment.

Also, (and this can be disputed since I cannot find where I read it) I recall seeing a study where the % of men believing WP is correct is at 70% while women believe it's accuracy to be only 40%. This does seem logical to me, since most of WP's subjects that indeed have to do with women is from the perspective of men. And fixing it can be practically impossible.

Age levels will slowly increase on Wikipedia as more individuals use phones over desk-top computers along with the aging code itself that's being used by the site. Plus, there are just way more interesting things to do now on the internet then in 2001.

[Earthy Astringent]

All of the alleged problems with the survey aside, how much do you think it would cost the WMF to hire a professional to create, execute, and analyze a "harassment" survey?

Why do they insist on doing everything in house? Are they being frugal, or is it hubris?

[Randy from Boise]

This indicates that they actually DID attempt to randomize participation, which means something else is going on with the data.

What did the invitation say?

It's one thing to say you're conducting a survey across a probability sample, please participate.

It's another thing to say you're conducting a survey about the harassment we experience on Wikimedia projects, please participate.

I honestly don't remember the banner but of course your observation about the wording is on-point.

That's actually one reason I think this is magnificent as a snapshot of gender and age: it wasn't marketed as such so the silly answer trolling on those matters was probably nil. The whole multiple categories of "other gender" thing was pretty silly but if you toss out that 3 percent of "undefinables," that still leaves a very damned fine gender breakdown.

Before somebody starts a PC rant on me, please do note that I think that Trans-M and Trans-F are fine and valid gender categories — both of which registered at zero on the survey. It is the Differently Abled Other Genderqueer and Intermediate Gender Personages to which I refer. (Yeah, good luck with that on your income tax forms...)

Anyway, yeah...

RfB

[Randy from Boise]

...you certainly cannot draw any quantitative conclusions from this nearly useless dataset.

They very accidentally produced something useful about gender and age. That is all.

RfB

[Earthy Astringent]

I'm wondering if any of you have ever actually spent any time on any forums where the majority of the users are women? 'Cause overall, it's much, much less of what I can only classify as a "war-like" environment.

I have, and while direct aggressive "fuck you" type behavior is non-existent, there are other noticeable differences:

The amount of passive aggressive behavior is off the charts when a spat breaks out, and battlesides are quickly drawn and a moderator has to lock the topic,

If a someone makes an argument against the house POV, the house will hen-peck the dissenter if they fail to come around. And I intentionally used "hen peck" because once blood appears everyone wants to get a taste. On the rare occasion that the dissenter is able to prove they are "right" (borrowing from Graham's hierarchy of arguments), one of the Alphas will announce "I'm ignoring X!", and the greek chorus yells "me too!"

Overemphasis: Excessive use of caps lock, bolding, underlining (ugh), and coloring. For some reason it's usually older people who use this tactic.

[Kelly Martin]

All of the alleged problems with the survey aside, how much do you think it would cost the WMF to hire a professional to create, execute, and analyze a "harassment" survey?

Why do they insist on doing everything in house? Are they being frugal, or is it hubris?

Hubris. Wikipedia is infected with the mad belief that what they are doing is something that humanity has never tried before. Because of this, the old rules do not apply and thus anyone's experience with other organizations is irrelevant, and nobody outside of The Movement can truly understand The Movement, and thus only members of The Movement can provide meaningful insights into how The Movement can better meet its goals.

[Randy from Boise]

All of the alleged problems with the survey aside, how much do you think it would cost the WMF to hire a professional to create, execute, and analyze a "harassment" survey?

Why do they insist on doing everything in house? Are they being frugal, or is it hubris?

Hubris. Wikipedia is infected with the mad belief that what they are doing is something that humanity has never tried before. Because of this, the old rules do not apply and thus anyone's experience with other organizations is irrelevant, and nobody outside of The Movement can truly understand The Movement, and thus only members of The Movement can provide meaningful insights into how The Movement can better meet its goals.

.........and confidence intervals be damned!

Things have really been distorted by the fact that there is a lot of money in play all of the sudden and there are careers to be made with an up and coming "software company" called WMF.

I really have no clue why they seem unable to see their limitations. Anyone skimming to page 17 could immediately see that the output of this survey was retarded garbage — yet, they can't?

My own mistake was thinking that the retarded garbage was produced on purpose to push a POV rather than being the distilled extract of their software incompetence — which I, ummm, retrospect was a more likely scenario all along.

RfB

[Poetlister]

One thing that bears mentioning is that this survey somewhat overrepresents English-Wikipedia. About 45% of respondents came from En-WP, compared to something around a 35% share of Very Active Wikipedians each month.

RfB

Maybe there's more harassment on the English Wikipedia than elsewhere. From my experience on the French and German sites, I think that may be the case. Similarly on the age distribution, maybe older people get harassed more, or feel worse about it if they are harassed.

[SneakySasha]

I'm wondering if any of you have ever actually spent any time on any forums where the majority of the users are women? 'Cause overall, it's much, much less of what I can only classify as a "war-like" environment.

I have, and while direct aggressive "fuck you" type behavior is non-existent, there are other noticeable differences:

The amount of passive aggressive behavior is off the charts when a spat breaks out, and battlesides are quickly drawn and a moderator has to lock the topic,

If a someone makes an argument against the house POV, the house will hen-peck the dissenter if they fail to come around. And I intentionally used "hen peck" because once blood appears everyone wants to get a taste. On the rare occasion that the dissenter is able to prove they are "right" (borrowing from Graham's hierarchy of arguments), one of the Alphas will announce "I'm ignoring X!", and the greek chorus yells "me too!"

Overemphasis: Excessive use of caps lock, bolding, underlining (ugh), and coloring. For some reason it's usually older people who use this tactic.

Wow! I must know the nicest places on the 'net!

The amount of passive aggressive behavior is off the charts when a spat breaks out, and battlesides are quickly drawn and a moderator has to lock the topic,

If a someone makes an argument against the house POV, the house will hen-peck the dissenter if they fail to come around. And I intentionally used "hen peck" because once blood appears everyone wants to get a taste. On the rare occasion that the dissenter is able to prove they are "right" (borrowing from Graham's hierarchy of arguments), one of the Alphas will announce "I'm ignoring X!", and the greek chorus yells "me too!"

Although, I fail to see how this description is any different from Wikipedia. And I couldn't believe the amount of gossip I encountered on Reddit!

My biggest annoyances would be sideways faces and "LOL". Whenever I see someone consistently using all caps, I tend to wonder if they're just far-sighted