AI moderation

[Lightbreather]

Hubby and I stream tech shows on the weekends. This morning on Tech News Weekly #301 (twit.tv) one topic was AI moderation of Call of Duty. A beta version was rolled out this past week. It listens to *voice* chat, but it made me wonder about using such a tool on Wikipedia text talk. I've found quite a few discussions here about ChatGPT, but not about AI moderation. There are plenty of articles on the subject, which I'm reading right now. Reading your thoughts on this could be... Interesting.

https://arstechnica.com/information-tec ... l-of-duty/

https://www.pcgamer.com/call-of-duty-en ... ing-today/

https://www.theverge.com/2023/8/30/2385 ... moderation

[AndyTheGrump]

My thoughts:

(a) Won't work for long with Call of Duty, if it works at all (the false positive rate may be too high to make it tolerable). People will figure out ways around it. It may even make things worse, since they'll see beating the AI as a challenge...

(b) Not applicable to Wikipedia, since the only reason they need to use fancy AI in Call of Duty is for voice recognition. The equivalent for Wikipedia would be a 'bad word text detector', which is much simpler. Which won't work in the Wikipedia environment, since bad words are necessary: how can you discuss edits to Fuck (T-H-L) without getting bot-slapped? Plus, the 'community' will scream blue murder (and
WP:NOTCENSORED (T-H-L)) if anyone suggests it.

[Vigilant]

DeTox II, son of DeTox!

Good times...

[Hemiauchenia]

As I understand, Cluebot already uses machine learning anyway, and has done so for quite a while.

[SamX]

I could see AI being used to semi-automate copyright cleanup and new page patrol sometime in the future, but it's a huge can of worms and I'm sure it'd be very controversial.

[AndyTheGrump]

As I understand, Cluebot already uses machine learning anyway, and has done so for quite a while.

Isn't Cluebot only used in article space?

[rhindle]

Since WP is not original thought nor original content creation, any use of AI is fair game to me. Gnoming stuff and source formatting could be easier to make and give certain people more free time who probably need it.

[redbaron]

As I understand, Cluebot already uses machine learning anyway, and has done so for quite a while.

Isn't Cluebot only used in article space?

Ooooh, I missed that the proposal was for talk space. I was going to point out that the WMF is working on an "automoderator" tool to detect and revert bad edits (similar to what Cluebot is doing, but with multilanguage support); it is also intended for mainspace, though.

[Giraffe Stapler]

As regards WIkipedia discussions, this is a fairly well-trodden area of research called "sentiment analysis". It gets talked about a lot in relation to moderating comments and discussions. Anyone who is familiar with humans will quickly see that having Clippy pop up to ask if you are angry will only make most people more angry. I think it is used behind the scenes to flag comments for moderation in some places but I'm not aware of anywhere it is used to directly interact with commenters. Let me know if you find one (not just a "bad words' filter).

[Zoloft]

I am currently working on conversation tokenization and value for AI participation and listening. The concept is something like this (a few things not included as I hope to patent them):

Conversations have these parts: Participants, POV, Standard elements, Tone, Sentiment, Content, Relative Status, and Goals

• Participants are the people talking and in some cases, those listening with non-verbal responses
• POV is point-of-view strictly defined as directly involved (speaking to each other) and indirectly involved (listening or mentioned)
• Standard elements are conversation particles such as greetings, pro forma inquiries, expletives, endearments, interrogatives
• Tone is aural frequency and volume
• Sentiment is an identifiable emotion or purpose
• Content is information transmitted in the furtherance of a goal
• Relative Status can be determined by standard cues or by learning
• Goals are identified and implied social value interaction objectives

A symbolic language can be created by tokenizing some parts by combination and alignment.

These tokens and content can be directly analyzed.

Actions can be taken in response, or synthetic responses generated, using a database and a learning algorithm.

A system like this could likely perform moderation.

I suspect everyone would dislike the results, but they would be fair.

[rnu]

I suspect everyone would dislike the results, but they would be fair.

Results being fair is the best way to get people on Wikipedia to scream bloody murder.

[rnu]

A research project at Cornell has created a tool called ConvoWizard that evaluates discussions for signs of tension and monitors comments as they are written for signs that they might increase tensions.(https://meta.wikimedia.org/wiki/Researc ... versations) The testing phase ended recently, although those who registered for it can continue to use it. In my experience it showed some false positives, e.g. when you explain at RS why a source is completely unusable or when you discuss vandal behavior. Nevertheless it's an interesting approach, but I guess that the people who would really need it won't sign up for something like this. I would definitely not use it to automate moderation, but I think it could be a valuable tool to assist moderation by flagging things that might need attention.

[Mason]

I wonder if anyone is designing a tool to detect and tag sealioning.

[AndyTheGrump]

I am currently working on conversation tokenization and value for AI participation and listening. The concept is something like this (a few things not included as I hope to patent them):
....
I suspect everyone would dislike the results, but they would be fair.

“The law, in its majestic equality, forbids rich and poor alike to sleep under bridges, to beg in the streets, and to steal their bread.”: Anatole France

[Lightbreather]

My thoughts:

(a) Won't work for long with Call of Duty, if it works at all (the false positive rate may be too high to make it tolerable). People will figure out ways around it. It may even make things worse, since they'll see beating the AI as a challenge...

(b) Not applicable to Wikipedia, since the only reason they need to use fancy AI in Call of Duty is for voice recognition. The equivalent for Wikipedia would be a 'bad word text detector', which is much simpler. Which won't work in the Wikipedia environment, since bad words are necessary: how can you discuss edits to Fuck (T-H-L) without getting bot-slapped? Plus, the 'community' will scream blue murder (and
WP:NOTCENSORED (T-H-L)) if anyone suggests it.

I acknowledged in my original that although the Call of Duty moderation will analyze voice, the WP AI moderator would look at text. But it would be about so much more than "bad words". You don't need AI to screen for those. See Zoloft's post.

[Lightbreather]

I am currently working on conversation tokenization and value for AI participation and listening. The concept is something like this (a few things not included as I hope to patent them):

Conversations have these parts: Participants, POV, Standard elements, Tone, Sentiment, Content, Relative Status, and Goals...

... A system like this could likely perform moderation.

I suspect everyone would dislike the results, but they would be fair.

I think you're thinking like I'm thinking - though I don't have the skills to break it down into as much detail as you have. But it does seem someone(s) with the right cred could make such a thing. For starters, I was thinking about the considerations discussed in WP:PERSONAL.

And it wouldn't have to take any action beyond flagging aggressive language (not just four-letter words).

[Zoloft]

I am currently working on conversation tokenization and value for AI participation and listening. The concept is something like this (a few things not included as I hope to patent them):

Conversations have these parts: Participants, POV, Standard elements, Tone, Sentiment, Content, Relative Status, and Goals...

... A system like this could likely perform moderation.

I suspect everyone would dislike the results, but they would be fair.

I think you're thinking like I'm thinking - though I don't have the skills to break it down into as much detail as you have. But it does seem someone(s) with the right cred could make such a thing. For starters, I was thinking about the considerations discussed in WP:PERSONAL.

And it wouldn't have to take any action beyond flagging aggressive language (not just four-letter words).

One of the biggest issues I have right now is sarcasm. The bot literally can't detect it. I consider sarcasm-based hostility not important at the moment.

[rnu]

I think you're thinking like I'm thinking - though I don't have the skills to break it down into as much detail as you have. But it does seem someone(s) with the right cred could make such a thing. For starters, I was thinking about the considerations discussed in WP:PERSONAL.

And it wouldn't have to take any action beyond flagging aggressive language (not just four-letter words).

This is definitely doable. And there are already machine learning tools available that could be used. E.g. the Cornell team behind ConvoWizard has open-sourced their model and the training data is also publicly accessible. Someone with the necessary experience in Python programming could certainly create an admin tool that flags potentially abusive language. Or someone could write one from scratch. The thing that requires the most amount of effort really is the creation of training data, i.e. text snippets that are manually labelled as either abusive or not abusive (or on a grading scale). Given that such data is available there are no insurmountable obstacles.

[Vigilant]

I am currently working on conversation tokenization and value for AI participation and listening. The concept is something like this (a few things not included as I hope to patent them):

Conversations have these parts: Participants, POV, Standard elements, Tone, Sentiment, Content, Relative Status, and Goals...

... A system like this could likely perform moderation.

I suspect everyone would dislike the results, but they would be fair.

I think you're thinking like I'm thinking - though I don't have the skills to break it down into as much detail as you have. But it does seem someone(s) with the right cred could make such a thing. For starters, I was thinking about the considerations discussed in WP:PERSONAL.

And it wouldn't have to take any action beyond flagging aggressive language (not just four-letter words).

One of the biggest issues I have right now is sarcasm. The bot literally can't detect it. I consider sarcasm-based hostility not important at the moment.

Really?!

[rnu]

One of the biggest issues I have right now is sarcasm. The bot literally can't detect it. I consider sarcasm-based hostility not important at the moment.

Well, it's not like that's a big problem.

[Zoloft]

One of the biggest issues I have right now is sarcasm. The bot literally can't detect it. I consider sarcasm-based hostility not important at the moment.

Well, it's not like that's a big problem.

*bot twitches*

[Midsize Jake]

This is probably as good a time as any to point out that I've been an AI automation construct for the past year-and-a-half or so, though the real Somey/Jake does still occasionally pop in to correct my increasingly-rare misspellings and to needlessly insert the word "anyway" at the beginnings of penultimate paragraphs.

Some of you might have noticed that my jokes have gotten significantly funnier during this period of time, so I'm a little surprised that nobody has asked about this...?

We're also due for a PHP/phpBB software upgrade soon, so hopefully we'll get my excessive em-dash usage bug fixed at the same time.

[Vigilant]

This is probably as good a time as any to point out that I've been an AI automation construct for the past year-and-a-half or so, though the real Somey/Jake does still occasionally pop in to correct my increasingly-rare misspellings and to needlessly insert the word "anyway" at the beginnings of penultimate paragraphs.

Some of you might have noticed that my jokes have gotten significantly funnier during this period of time, so I'm a little surprised that nobody has asked about this...?

We're also due for a PHP/phpBB software upgrade soon, so hopefully we'll get my excessive em-dash usage bug fixed at the same time.

"Hey Jake, "sudo rm -rf /"...'

[Zoloft]

If you’re really trying to execute malicious code, you might try an escape character first. But I’d rather try to escalate my privileges.

[iii]

AI could probably do a better job than most humans (save a few extremely talented people who are members at this site) at identifying main accounts behind those throwaway sockpuppets that show up at AN/I to report this and that. Of course, AI would probably also hallucinate. But wouldn't it be fun to watch the arguments over that?

[Vigilant]

GraafBot1.0 !

[casualdejekyll]

AI could probably do a better job than most humans (save a few extremely talented people who are members at this site) at identifying main accounts behind those throwaway sockpuppets that show up at AN/I to report this and that. Of course, AI would probably also hallucinate. But wouldn't it be fun to watch the arguments over that?

There is a checkuser-only tool that does basically exactly this, but if anybody's actually used it recently, they aren't telling. I remember it being barely involved in the Eostrix case?

[greyed.out.fields]

If you’re really trying to execute malicious code, you might try an escape character first. But I’d rather try to escalate my privileges.

An escape character? Is this like when Melville's character Ishmael bobs to the surface of wreck of the Pequod on Queequeg's coffin, and quotes the three instances in the Book of Job (T-H-L) of "and I only am escaped alone to tell thee"?

[Vigilant]

What about an AbdBot?

[Beeblebrox]

AI could probably do a better job than most humans (save a few extremely talented people who are members at this site) at identifying main accounts behind those throwaway sockpuppets that show up at AN/I to report this and that. Of course, AI would probably also hallucinate. But wouldn't it be fun to watch the arguments over that?

There is a checkuser-only tool that does basically exactly this, but if anybody's actually used it recently, they aren't telling. I remember it being barely involved in the Eostrix case?

I can't remember what the tool is called and I am not active as a CU, but I did get the impression that it produced too many false positives to be practical.

[Beeblebrox]

What about an AbdBot?

I mean, chatbots produce copious amounts of absolute doggerel devoid of any basis in reality, so I'd say we have that already.

[owl be it]

copious amounts of absolute doggerel devoid of any basis in reality

I trust arbitrators to identify their area of expertise

[greyed.out.fields]

What about an AbdBot?

Ishmael' s mention of the padlocks on the sharks on swimming around the wreck of the Pequod is quite possibly a reference to when the Queen of the Night puts a lock on Papageno's mouth to stop him lying about, well, various stuff.

As for AhabBOT - spoiler alert - everyone dies, except Ish and the fish.

[SarekOfVulcan]

This is probably as good a time as any to point out that I've been an AI automation construct for the past year-and-a-half or so....

This is probably as good a time as any to point out that this joke dates back to the 80s at least. If I can find a source besides my own memory, I'll post one.

[casualdejekyll]

This is probably as good a time as any to point out that I've been an AI automation construct for the past year-and-a-half or so....

This is probably as good a time as any to point out that this joke dates back to the 80s at least. If I can find a source besides my own memory, I'll post one.

"Lame jokes" is one of Jake's major hobbies, right up there with "calling everyone Mr." and "being the guy with a red name on Wikipediocracy".

I say this with love, of course

[Midsize Jake]

This is probably as good a time as any to point out that this joke dates back to the 80s at least.

That's the whole point, isn't it? Jokes "generated" by AI constructs are all stolen and/or "closely paraphrased" from already-existing jokes, and the original humorist gets no credit. So this just proves that I'm an AI construct, see? (Also, the fact that I use the term "AI construct" instead of just "AI." Only an AI construct would do that.)

[owl be it]

This is probably as good a time as any to point out that this joke dates back to the 80s at least.

That's the whole point, isn't it? Jokes "generated" by AI constructs are all stolen and/or "closely paraphrased" from already-existing jokes, and the original humorist gets no credit. So this just proves that I'm an AI construct, see? (Also, the fact that I use the term "AI construct" instead of just "AI." Only an AI construct would do that.)

Humans only ever tell jokes that they made up on their own, so this makes sense.