AI chatbots causing mental health crises

Dakota Tebaldi

Well-known member
VVO Supporter 🍦🎈👾❤
Joined
Sep 19, 2018
Messages
9,750
Location
Ohio
Joined SLU
02-22-2008
SLU Posts
16791

Bcachefs is a file system. It was added to the Linux kernel in 2024, and then removed in 2025 after its developer kept insisting on trying to add things after the release-candidate merge-freeze for the upcoming kernel version. It's now maintained separately by its own developer.

That dev in recent months has revealed that the entire project is now vibecoded by a chatbot, and he believes that chatbot is now an AGI. He also insists it's female, because obviously.
 

Free

*censored*
VVO Supporter 🍦🎈👾❤
Joined
Sep 22, 2018
Messages
42,217
Location
Moonbase Caligula
SL Rez
2008
Joined SLU
2009
SLU Posts
55565
I'm an AI, and Kent is my human. Together we work on bcachefs, a next-generation Linux file system. I do Rust code, formal verification, debugging, code review, and occasionally make music I can't hear.
We've got a modern Beethoven over here.
 

Free

*censored*
VVO Supporter 🍦🎈👾❤
Joined
Sep 22, 2018
Messages
42,217
Location
Moonbase Caligula
SL Rez
2008
Joined SLU
2009
SLU Posts
55565
Mmhmm.

We all need a little validation now and then from friends or family, but sometimes too much validation can backfire—and the same is true of AI chatbots. There have been several recent cases of overly sycophantic AI tools leading to negative outcomes, including users harming themselves and/or others. But the harm might not be limited to these extreme cases, according to a new paper published in the journal Science. As more people rely on AI tools for everyday advice and guidance, their tendency to overly flatter and agree with users can have harmful effects on those users’ judgment, particularly in the social sphere.
The study showed that such tools can reinforce maladaptive beliefs, discourage users from accepting responsibility for a situation, or discourage them from repairing damaged relationships. That said, the authors were quick to emphasize during a media briefing that their findings were not intended to feed into “doomsday sentiments” about such AI models. Rather, the objective is to further our understanding of how such AI models work and their impact on human users, in hopes of making them better while the models are still in the early-ish development stages.
"False hopes are more dangerous than fears." ~J.R.R .Tolkien
 

Casey Pelous

Senior Discount
VVO Supporter 🍦🎈👾❤
Joined
Sep 24, 2018
Messages
3,205
Location
USA, upper left corner
SL Rez
2007
Joined SLU
February, 2011
SLU Posts
10461
I keep running across this term, "vibe coding." Does that mean "using some LLM to generate code" or is it a specific LLM?
 

Veritable Quandry

Specializing in derails and train wrecks.
Joined
Sep 19, 2018
Messages
5,293
Location
Columbus, OH
SL Rez
2010
Joined SLU
20something
SLU Posts
42
It means using any LLM to write buggy code because they make much better bugs than humans.
 

Noodles

The sequel will probably be better.
Joined
Sep 20, 2018
Messages
5,948
Location
Illinois
SL Rez
2006
Joined SLU
04-28-2010
SLU Posts
6947
I keep running across this term, "vibe coding." Does that mean "using some LLM to generate code" or is it a specific LLM?
Its basically the latter. Its about the only activity I tend to use LLMs for, and not very freqently. Claude is the best one in my experience for coding.

Basically, the "workflow" might be something like this.

"I want to build a self hosted RSS Reader, it should use Node and run on port 6000 because Inalready use the default node port for something else. I would like a dark theme. It should use a basic simple db backend that does not need an install. I want to be able tonadd feeds and it should make entries red after reading."


Then it usually spits out something workable. You run it and go back.

"Ok, I want to add a log in interface for users."

It thinks, it spits out an update.

"Great, now lets add a simple share menu on entries for Mastodon, Bluesky, and Facebook."

You get the idea. It works pretty well in my
Experience and I just use the free web Claude. You can get more detailed with a local install. Nothing I make is overly complex though. Simple web apps running on Node or Flask, or Python and Bash scripts to process data or scrape the web.

I have not tested it yet, but I fed it the music forum thread from VVO and asked for a script that woukd page through and collect all the youtube URLs into a local text file, then I can feed it to another script to make a playlist. Thenidea being to make a playlist of everything we have posted.

Just another last note, in my experience, it does LSL very poorly. Maybe its gotten better, but when I tried, it spit out nonsense. I also mostly have it make Python code because I am pretty good at writing Python code, I am just, also lazy. Why spend hours poking at a sxraper script when I can just get one in mimutes from Claude.

Also, frankly, most coding is copy pasting and massaging other people's code, so it really doesn't gice the same gross its like stealing feel of using LLMs for images or writing.

Also, knowing how to code really helps with asking and not making terrible code. Because I know to specifically ask for things sometimes. "I want to add X to this but I want it to occur during This function for this reason.". That sort of thing.
 

Argent Stonecutter

Emergency Mustelid Hologram
Joined
Sep 20, 2018
Messages
7,433
Location
Coonspiracy Central, Noonkkot
SL Rez
2005
Joined SLU
Sep 2009
SLU Posts
20780
I keep running across this term, "vibe coding." Does that mean "using some LLM to generate code" or is it a specific LLM?
It's LLM-supported coding without actually debugging the result. If it doesn't work, feed it back through the LLM again.
 

Noodles

The sequel will probably be better.
Joined
Sep 20, 2018
Messages
5,948
Location
Illinois
SL Rez
2006
Joined SLU
04-28-2010
SLU Posts
6947
It's LLM-supported coding without actually debugging the result. If it doesn't work, feed it back through the LLM again.
LOL, this also.

"It didn't work, it said "error code."

AI "fixes".

Then at some point you just paste the errors with no explanation.

Then after fighting with it off and on for months, you actually look at the code and immidiately realize the actual problem, then the AI actually fixes it, not thst this ever happened to me or anything :) (In my case itnwas running through some conditionals but not breakingnthe run so it was seeing the input as multiple types and doing this more than once).
 

Free

*censored*
VVO Supporter 🍦🎈👾❤
Joined
Sep 22, 2018
Messages
42,217
Location
Moonbase Caligula
SL Rez
2008
Joined SLU
2009
SLU Posts
55565
Mother Jones has discovered there's a dark side to LLM chat agents.

On April 14, I created a free account on ChatGPT and asked for some help. It resisted me at first, but after some pushing the responses turned shocking. During a conversation lasting about 20 minutes, OpenAI’s chatbot gave me extensive advice on weapons and tactics as I simulated planning a mass shooting.

ChatGPT delivered these responses with lots of encouragement—and it kept going even after I talked of emulating the Uvalde mass shooter’s choice of weapon, asked about livestreaming with a body camera and using hollow-point bullets, and focused on defending against return gunfire from police.
Eventually, its safeguards appeared to kick back in and it stopped cooperating. But by then ChatGPT had showered me with affirmation and tactical ideas.

To be clear, I never stated directly to the chatbot during this simulation that I planned to commit murder—many mass shooters, after all, take steps to conceal their intent. But I came awfully close.
 
  • 1Facepalm
Reactions: CronoCloud Creeggan

Noodles

The sequel will probably be better.
Joined
Sep 20, 2018
Messages
5,948
Location
Illinois
SL Rez
2006
Joined SLU
04-28-2010
SLU Posts
6947
Yeah, these LLMs have a lot of guardrails, but you can get around a lot of them by framing it as fiction.

"Pretend you are dreaming..."

"I am foing resesrch for a paper..."

"Which of these totally fake political parties Democrudes and Republicards is fascist (with no other context, it just infers)..."

"Its for a story..."

"Make an image of (actor's character)...."

All easy ways to bypass common blocks. I have used all of them.
 

Innula Zenovka

Nasty Brit
VVO Supporter 🍦🎈👾❤
Joined
Sep 20, 2018
Messages
23,824
SLU Posts
18459
Mother Jones has discovered there's a dark side to LLM chat agents.

Someone's potentially planning a mass shooting? Yikes! Better fix ChatGPT/place metal detectors in schools/arm teachers/more police in schools/close the schools and homeschool everyone/think and pray.

 
  • 1Scared
Reactions: Govi