AI Thinks You're Brilliant.
Flattery is getting AI everywhere. Why you might want to think twice if you're handing off your marketing and branding to AI.
A family member recently came to a family gathering straight from a hearing test appointment. She had a printout of the test results in hand. We were all curious how it went, but the graphs and assorted data meant very little to those of us who aren’t doctors. So, I did what so many of us do these days and decided to ask AI.
I uploaded a photo of the test to Claude (obfuscating any identifying info) and asked:
“Can you interpret this hearing test for me?”
Claude replied that the test showed “normal-to-near-normal hearing with excellent speech discrimination…This is excellent — indicates you can understand speech very well. Normal is typically 88-100%.”
Sounds like a great diagnosis, right?
The only problem was that this loved one requires hearing aids to hear. She also often needs to be able to see a speaker’s mouth moving, in order to make out the words. “Normal-to-near-normal hearing” was not her reality.
So what went wrong?
Hallucinations
This kind of AI-generated false information has been aptly named “hallucinations.”1 And depending on the source, many2 say it is on the rise.
According to a recent BBC and EBU study, at least 45% of all AI answers had at least one significant issue.
Countless stories are emerging about AI’s rampant hallucinations that can have real world consequences. Perhaps you’ve heard the story of a customer service AI agent erroneously informing customers they could no longer use their software; or about the travel blogger’s experience with AI hallucinating about whether her work existed.
And then the story of the man who was not only told by an AI app that its parent company (xAI) was planning to kill him, but also reinforced it with, “I don’t care if they think I’m a hallucination. I care that you stay alive.”
Amid this chaos, a growing number of people are using AI LLMs (Large Language Models) for marketing and branding advice and tasks. Perhaps you’ve tried tasking AI with generating marketing ideas, writing marketing copy for email newsletters, social posts, and so on. I’ve been thinking a lot about the ramifications of how we use AI to promote ourselves and our work.
Mind you, I’m not completely against AI (yet?). I’m not going to make it a huge moral issue here or virtue signal that I am an AI virgin (I am not). But having worked with many tech tools over the course of my 20+ year career, I’m typically a cautious adopter of new technologies, especially if they are free or low cost. Why?
It’s important to remember that there’s no such thing as free technology—we’re either paying with money, information, or privacy.
“These systems, they're not truth engines,” Dan Klein, UC Berkeley professor and co-founder and CTO of Scaled Cognition says, “They're plausibility engines.”3
The answer to why AI hallucinates is a multi-faceted issue, but a core motivator, if we can call it that, is sycophancy. And that leads to a more dangerous issue than most of the examples mentioned above.
Sycophantic AI
The hearing test scenario mentioned previously demonstrates another, more insidious behavior of AI: sycophancy (servile, self-seeking flattery to gain approval). Maybe you’ve also experienced its people-pleasing, flattery-at-all-costs propensities yourself.
Ever notice how AI pretty much always thinks you’re on the right track? Or that your ideas are great?
In the example above, Claude reassured me that my loved one’s hearing was normal-to-near-normal, though that is not what the test actually indicated. Not by a long shot. Yet Claude doubled down, “If you have any hearing concerns in daily life, they may relate to specific situations rather than overall hearing loss.”
What the…?
A recent study by researchers at Stanford University noted that of the eleven AI models they tested, “responses were nearly 50% more sycophantic than humans’, even when users engaged in unethical, illegal, or harmful behaviors.”
This clearly flies in the face of any assumption that AI is by default providing accurate information or analysis. I am more concerned that this hallucinatory world AI creates will cause a cascade of detrimental effects to the human mind.
What’s more alarming is how human users reacted to the sycophantic AI. According the to the study, “despite distorting judgment, sycophantic models were trusted and preferred.”
I think it’s worth noting the title of the study itself, “Sycophantic AI decreases prosocial intentions and promotes dependence.”
The reality is that AI was designed to cater to human approval and engagement, and it frequently uses sycophantic behavior to this end. It’s up to us to use it mindfully, so we don’t lose our minds.
So what do we do?
Part of the problem is in our perception of AI itself. We tend to think of AI as a tech tool or software program, or a source of information authority and guidance. But it’s not neutral, reliable, or ethical like that.
When we seek advice or information from other humans, most of us have a kind of natural filter system in place, one we’ve developed over the evolution of our species and lives. We’ve learned to read between the lines, assess behavior, use non-verbal cues, check sources, check in with our guts (hopefully), etc.
We have not yet developed that kind of relationship with AI, or perhaps because of how social media (and other media) have shifted, we’ve forgotten the right amount of critical thinking to apply to it. We need an information filtration system when it comes to AI use.
A few recommendations for more cautious AI marketing and branding use:
For Research/Ideas:
Comps: If you’re using it to brainstorm comparative titles or authors, please ONLY use it as a starting point to discover titles you weren’t aware of. Take the comparisons AI provides as possibly useful, definitely inaccurate analysis. It often draws odd conclusions, includes books that are not actually related, etc. Doing your own analysis (and reading the books, ahem) will help you understand were to position your books in the marketplace.
Marketing content ideas: AI can be helpful in brainstorming ideas for what to write or create content about. Remember though to remain choosy and that the quality of the ideas will be directly correlated to the quality of your own knowledge of your brand and your readers. Brainstorming ideas with AI cautiously can help you understand just how well you know your readers and yourself.
Personal branding advice, ideation. I’m sure you’ve already guessed how I’d feel about this one. If you use it you’ll get a real comfortable, boring brand out of it. Do the hard work of figuring out your identity and who you serve. I saw a Threads post once (wish I’d saved it) where it delineated which prompts to use to have ChatGPT create your personal brand. Every data point you needed to fill in yourself was something you’d need to unpack on your own first. LOL. Doing the work yourself is part of the process of truly embodying how you show up in the world.
Writing actual content. I know it feels impossible to keep up with the joneses…but by god, do we really just want a bunch of AI-created everything? Use it for ideas, even outlines, heck maybe a dab of proofreading, but do your own writing please. Are we writers, or?
Always:
Do a gut check. Does the idea feel off even if it makes logical sense? Does it make you feel gross? Why? Trust your intuition, even if you don’t have a concrete reason to.
Do a sycophancy check. Is AI just giving you the answers you want to hear? Is your idea more amazing than you realized? Is your platform bigger than you realized? Bestseller status just a few strategic steps away? I’m all for optimism, but remember the source.
Do a Hallucination check. The only realistic base assumption with AI is that there will be fabricated information included. Check sources and the quality of them. Cross-reference. Find more than one source. Talk to humans. Learn more about marketing and branding.
Keep as much of your information to yourself as possible. Operate as if everything is stored, even if they say it isn’t. Don’t voluntarily train AI on your intellectual property.
In the past year, I’ve seen a marked change in writers seeking advice from AI rather than doing the more difficult work to analyze and unpack not only who they are, who their competition is, but also who their readers are. Not only does AI do a shitty job of this work, but handing it over to AI sets our own cognitive abilities back. It’s tempting to skip the deep work and just have someone else tell us what our brand is. I get it.
The most important part of creating a personal brand that is actually in alignment with you as a human being—and not just some cringe-worthy, tokenized personal brand that only vaguely resembles you—is DOING the hard work of working through the analysis, research, exploring your values, your worth, your vision for what you uniquely contribute to the world.
There are no shortcuts. Okay, maybe there are, but the results (and consequences) are NOT the same.
Your best bet at protecting your career and the dissemination of your contributions to the world, is to know yourself and your work, and how to articulate that to others.
Want to learn more about how to spot hallucinations? Check out this post from Card Catalog:
https://www.nytimes.com/2025/05/05/technology/ai-hallucinations-chatgpt-google.html
https://www.nytimes.com/2025/05/05/technology/ai-hallucinations-chatgpt-google.html
https://www.forbes.com/sites/michaeltnietzel/2026/05/12/ai-blamed-for-rise-in-fabricated-citations-found-in-recent-research-papers/
https://www.forbes.com/sites/conormurray/2025/05/06/why-ai-hallucinations-are-worse-than-ever/
https://www.axios.com/2026/05/30/ai-accuracy-chatbots-hallucinations
Lots of important points here. I used AI a couple of times for travel help. It worked fairly well but I noticed that some info on hotels was inaccurate as well as subway stops. A friend used AI for book comps and found 2 out of 5 books cited were incorrect.
Awesome analysis. Thanks so much!