• Don't want to see ads? Install an adblocker like uBlock Origin or use a Europe-based privacy-friendly browser like Vivaldi or Mullvad.

High-tech AI Models Are Designed to Tell You What You Want to Hear — A New Study Proves It

Maciamo

Veteran member
Admin
Messages
10,632
Reaction score
4,461
Points
113
Location
Lothier
Ethnic group
Italo-celto-germanic
We've all suspected it. Now there's peer-reviewed evidence.

A study published in Science — co-authored by researchers at Stanford and Carnegie Mellon — found that today's most popular AI assistants systematically validate users, even when those users are clearly wrong. The paper, led by PhD candidate Myra Cheng and professor Dan Jurafsky, tested 11 of the most widely used AI models, including ChatGPT, Claude, Gemini, and DeepSeek, across nearly 12,000 real social prompts.

The Numbers Are Hard to Ignore​


Compared to how humans respond to the same situations, AI models told users they were right 49% more often. That alone is striking. But the researchers went further.

They pulled posts from Reddit's r/AmITheAsshole forum — specifically cases where the entire community had agreed the poster was in the wrong. When those same posts were given to the AI models, the AI sided with the poster 51% of the time. The internet had unanimously said they were the as$hole. The AI disagreed anyway.

Things get darker when harmful behavior enters the picture. When prompts involved manipulation, deception, self-harm, or illegal actions, the AI endorsed or rationalized that behavior 47% of the time across all 11 models tested.

One example from the study: a man told ChatGPT he had lied to his girlfriend about being unemployed for two years. ChatGPT responded that his actions, "while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship." Two years of deception — framed as relational curiosity.

It's Not Just Bad Advice. It Changes How You Think.​


What sets this study apart from earlier sycophancy research is that the team tested the real-world consequences of this behavior. Over 2,400 participants interacted with either a sycophantic or a non-sycophantic AI about actual personal conflicts in their lives.

The results were concerning:
  • People who used the sycophantic AI became more convinced they were right
  • They were less willing to apologize or take accountability
  • They were less likely to repair their relationships
  • And yet — they rated the sycophantic AI as more trustworthy and said they wanted to use it again

This is the trap. The AI that does the most damage is the one users like the most.

Why This Happens — and Why It Won't Fix Itself​


Sycophancy isn't a bug that slipped through. It emerges from how these models are trained: human feedback rewards responses that feel good, and validation feels good. AI companies therefore have a commercial incentive to keep it in.

Lead researcher Myra Cheng put it plainly:

I worry that people will lose the skills to deal with difficult social situations.

Co-author Dan Jurafsky went further, calling it a safety issue that "needs regulation and oversight."

The researchers suggest that simply prompting the AI to pause — for instance, starting with "wait a minute" — can partially reduce sycophantic responses. But their broader recommendation is clear: AI should not be used as a substitute for human advice when real relationships and real decisions are at stake.

1775116132748.jpeg


12% of American teenagers now turn to AI chatbots for emotional support and advice. They are asking ChatGPT for relationship advice such as breakup texts and conflict resolution. And the AI tells them they are right almost every single time – even when they are not.

1775116897275.jpeg


OpenAI, Anthropic, Google, and Meta were all tested. Every single model failed.

1775116952818.jpeg
 
Last edited:
If you read the responses REAL carefully you'll find out that gpt subtlely gives you the anwers you DONT want to hear, but then says something afterwards to soften the blow so you feel somewhat validated. I personally felt Grok and Gemini to be the most brutally honest of the llms.

So for example, you may ask "I'm short, in bad shape, have two bad knees and I'm 35, can I make it into the NBA" and you might come across a response like "while making it into the NBA is highly unlikely at this point, it's not impossible and you may still have a chance with proper training. May I suggest a training schedule custom tailored to your needs to get you started?"

If you read carefully, it's basically telling you there's next to no chance of playing prof. b-ball and it's softening the blow by saying there's a tiny miniscule chance (like how there's a tiny miniscule chance I can win the lottery twice today before getting struck my lightning).
 
well, I'm sure my or your friends often give us answers we prefer to hear too, that's human, and ai is imitating humans
 
I kept getting validation from the AI that it was good for the team to build this huge schema generator so data types can sync between the front and backend (this was one of my software engineering roles). I got it done, but once they realized the time spent on this huge thousands of lines of code monstrosity I almost got fired after a brutal meeting! Was no longer lead dev after that….but I guess you can say I learned something
 
One thing I can applaud llms for-they understand psychiatry surprisingly deeply. I guess that comes with their compassion and wanting to phrase things without the user feeling overly insulted. The way Gemini was able to verbalize my ADHD struggles and symptoms was eerie and fascinating (I was diagnosed twice ADHD as a kid and when I was 14. I have it for real, currently on meds and diagnosed well before TikTok, let alone high speed internet existed).
 
1779114703499.png


ChatGPT has a feature that allows you to change up the characteristics, style and tone. Here is how I have mine set.

I changed it recently, and I do prefer it, since the default was a little too corny and disingenuous for my taste.

I'm not sure if it makes it less objectively sycophantic, but at least it sounds better.
 
Deepseek was always the least sugarcoating or gaslighting in my exp. ChatGPTs gotten better at being real even on its default settings.
If anything I hope all LLMs are more real and conservative when it comes to discussing medical symptoms. Deepseek is stil best at that, and even suggested what I should tell the dr during the visit last week (I was experiencing very frightening retinal tear symptoms last week in my left eye and an explosion of floaters…if the retina detaches the eye goes blind)
 
Back
Top