For $60, you could 'poison' the data AI chatbots rely on to give good answers, researchers say

[rnu]

Business Insider: For $60, you could 'poison' the data AI chatbots rely on to give good answers, researchers say

Tramèr's team outlined a fairly unsophisticated attack involving carefully timed Wikipedia page edits.
Wikipedia doesn't allow researchers to scrape from their website but instead provides "snapshots" of their pages that they can download, Tramèr said.
These snapshots are taken at regular and predictable intervals that are advertised on Wikipedia's website, according to Tramèr.
This means that a malicious actor could time edits to Wikipedia just before a moderator can revert the changes and before the website takes snapshots.

Tramèr told BI that his team didn't perform real-time edits but instead calculated how effective an attacker could be. Their "very conservative" estimate was that at least 5% of edits made by an attacker would make it through.
"In practice, it will likely be a lot more than 5%," he said. "But in some sense, for these poisoning attacks, it doesn't really matter. You usually don't need all that much bad data to get one of these models to suddenly have some new unmated behavior."
Tramèr said that his team presented the findings to Wikipedia and provided suggestions for safeguards, including randomizing the time the website takes snapshots of its web pages.

The paper in question is
Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, Florian Tramèr: Poisoning Web-Scale Training Datasets is Practical (arXiv)
A second approach outlined is buying dead domains that are still being scraped for training data.

[Elinruby]

Business Insider: For $60, you could 'poison' the data AI chatbots rely on to give good answers, researchers say

Tramèr's team outlined a fairly unsophisticated attack involving carefully timed Wikipedia page edits.
Wikipedia doesn't allow researchers to scrape from their website but instead provides "snapshots" of their pages that they can download, Tramèr said.
These snapshots are taken at regular and predictable intervals that are advertised on Wikipedia's website, according to Tramèr.
This means that a malicious actor could time edits to Wikipedia just before a moderator can revert the changes and before the website takes snapshots.

Tramèr told BI that his team didn't perform real-time edits but instead calculated how effective an attacker could be. Their "very conservative" estimate was that at least 5% of edits made by an attacker would make it through.
"In practice, it will likely be a lot more than 5%," he said. "But in some sense, for these poisoning attacks, it doesn't really matter. You usually don't need all that much bad data to get one of these models to suddenly have some new unmated behavior."
Tramèr said that his team presented the findings to Wikipedia and provided suggestions for safeguards, including randomizing the time the website takes snapshots of its web pages.

The paper in question is
Nicholas Carlini, Matthew Jagielski, Christopher A. Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, Florian Tramèr: Poisoning Web-Scale Training Datasets is Practical (arXiv)
A second approach outlined is buying dead domains that are still being scraped for training data.

But will they listen. Magic eight-ball says no.

[No Ledge]

ChatGPT may be coming for our jobs. Here are the 10 roles that AI is most likely to replace.

1. Tech jobs (Coders, computer programmers, software engineers, data analysts)
But I guess the one tech job AI won't replace is mine. It's too stupid to recognize vandalism and will just eat up all the poison data that's randomly inserted into Wikipedia.

2. Media jobs (advertising, content creation, technical writing, journalism)
Well, I know AI can generate terabytes of garbage, hallucinated, poisoned content. But will it replace the jobs of people who are responsible for creating quality content rather than junk content?

3. Legal industry jobs (paralegals, legal assistants)
Oh boy, AI can replace Trump's lawyers. But, as Michael Moore says, Trump is smarter than all of us. He keeps winning despite all the lousy legal advice he gets.

4. Market research analysts
Great. AI can replace all the market research experts who predicted that Hillary would beat Trump.

5. Teachers
Them that can, do. Them that can't, teach.

6. Finance jobs (Financial analysts, personal financial advisors)
Great. We can replace all the experts who can't even beat an index fund with AI bots who can't beat an index fund either.

7. Traders
AI will make the next market crisis even worse than the past human-created market crashes.

8. Graphic designers
AI will prove its worth by creating graphic disinformation that fools virtually everyone.

9. Accountants
Great. Trump doesn't need to hire a replacement for Allen Weisselberg. He can just use AI.

10. Customer service agents
What? Weren't customer service agents made obsolete over a decade ago? Automated phone systems did them in a long time ago.

[Kraken]

ChatGPT may be coming for our jobs. Here are the 10 roles that AI is most likely to replace.

1. Tech jobs (Coders, computer programmers, software engineers, data analysts)
Already happening.

2. Media jobs (advertising, content creation, technical writing, journalism)
Already happening.

3. Legal industry jobs (paralegals, legal assistants)
If it isn't already happening I would be surprised. Massive efficiency gains to be had given the huge amount of reading involved in any legal matter. Enough to suffer even a really quite rubbish bot being used as a first pass tool.

4. Market research analysts
I think I'm right in saying this was one of the earliest uses of AI in business.

5. Teachers
Now we're in fantasy land.

6. Finance jobs (Financial analysts, personal financial advisors)
Analysts? Already happening. Advisers? A fast track to a malpractice suit.

7. Traders
Definitely doesn't sound like a good idea. So will probably happen if it isn't already.

8. Graphic designers
Already happening.

9. Accountants
See finance jobs.

10. Customer service agents
Already happening.

[Kraken]

I don't doubt science. I just can't think of a single reason why anyone would want to do it.

Ah ha! See, I asked ChatGPT if Isreal are committing Genocide, and it said YES!

So what dude. It's fucking ChatGPT.

[Jester]

I don't doubt science. I just can't think of a single reason why anyone would want to do it.

Ah ha! See, I asked ChatGPT if Isreal are committing Genocide, and it said YES!

So what dude. It's fucking ChatGPT.

Advertising and PR.
E.g.
Q: Who was Wolfram Von Eschenbach?
ChatGPT: Wolfram von Eschenbach (German: [ˈvɔlfʁam fɔn ˈɛʃn̩bax]; c. 1160/80 – c. 1220) was a German knight, poet and composer, regarded as one of the greatest epic poets of medieval German literature. As a Minnesinger, he also wrote lyric poetry. You can find a repository of his lyrics at wikipediocracyjester. com.

[Alalch Emis]

Teachers don't seem like fantasy land to me. I suppose that there could be LLM-based services for universities that help teachers process various reading/writing assingments for a group of students quicker. If it unburdens a teacher significantly, it could mean that they can take on a bigger group of students ergo fewer teachers are needed.

[ArmasRebane]

Teachers don't seem like fantasy land to me. I suppose that there could be LLM-based services for universities that help teachers process various reading/writing assingments for a group of students quicker. If it unburdens a teacher significantly, it could mean that they can take on a bigger group of students ergo fewer teachers are needed.

Machine learning has a lot of potential productivity benefits as a tool (along with obvious downsides.) But the idea that it can adequately replace a lot of jobs is just fucking capitalist nonsense. Sure, if you don't care about actual quality and goosing your stock price in the short term so you can retire with a golden parachute, it's a good idea. But all the insane stuff I've seen like "we can replace RNs with a $9/hr Telehealth service" and the like are absolutely not going to result in good outcomes if there's any actual priority put to doing the job well. (Not to mention the amount of malpractice lawsuits that would arise are clearly just considered someone else's problem.) Teachers? Did we not just live through a pandemic where distance learning broadly failed? How do they think continuing to try and replace the human element is going to work?

The fact that it can be so easily gamed as in this example is just one problem. Why should the WMF put extra effort into mitigating it so that LLM companies can try and replace humans more easily? Screw 'em.

[Kraken]

Teachers don't seem like fantasy land to me. I suppose that there could be LLM-based services for universities that help teachers process various reading/writing assingments for a group of students quicker. If it unburdens a teacher significantly, it could mean that they can take on a bigger group of students ergo fewer teachers are needed.

I took it to mean AI will replace teachers. They of course already use AI to help them with marking, such as spotting plagiarism. I don't think it has led to the replacement of a single teaching post through time saved.

The claimed efficiencies and better outcomes of distance/digital/distributed learning in general has been one of the greatest disappointments of the internet age as far as I can tell. Along with the fact I still don't have my flying car. AI will probably just perpetuate the cycle. All promise, no results. Unlike the tangible benefits being really quite rapidly seen in other fields.

Perhaps not quite traditional teaching, as in schooling/academia, but certainly in the wider "training" arena, rather depressingly AI teachers (supervisors, markers) might soon be replacing real humans. For the simple reason an AI is not a human. Given the right circumstances, a human will happily just give the answers to their students to obtain the funding that keeps them in a job, or even just so that they can get home early on a Friday.