Skip to main content ↓
A person in a white shirt and blue helmet riding a bicycle on the edge of a concrete wall with a large yellow circle in the background, resembling a stylized sun against a blue sky.

8 ChatGPT Fails That Show the Dangers of Relying Too Much on AI

Let me start out by saying this: ChatGPT isn’t all bad. There are lots of things that it, and other generative AI tools like it, can be used for. In marketing, for example, it can be a great tool for helping generate things like title tags and email outlines.

But despite all its benefits, there are still a whole lot of problems inherent to the tool. Some of these are smaller kinks that OpenAI may work out eventually, but others are much more serious and deep-rooted. These problems are evident in the many, many recorded ChatGPT fails online.

A lot of these bad ChatGPT results are comical, but they still point to deeper issues. And others are scarier than they are funny. If you plan to use ChatGPT — be it for work or personal use — it would be smart to inform yourself of the various ways it can screw up.

There are countless ChatGPT mistakes already recorded out there, but on this list, I’m going to focus specifically on eight problems that often crop up in the tool. I’ll also look at examples of each one. Those problems include:

  1. Not adhering to word limits
  2. Failing at simple math and logic
  3. Not grasping humor
  4. Struggling to generate new ideas
  5. Falsifying sources
  6. Lying about its own protocols
  7. Hallucinating fake information
  8. Producing biased responses

Keep reading to see some examples of major ChatGPT failures. Then, if you’re interested in learning more about the digital world — particularly digital marketing — be sure to subscribe to our email newsletter, Revenue Weekly!

Don’t miss our Marketing Manager Insider emails!

Join 200,000 smart marketers and get the month’s hottest marketing news and insights delivered straight to your inbox!

Enter your email below:

Inline Subscription Form – CTA 72

*” indicates required fields

Hidden
Hidden
Hidden
Hidden
Hidden
Hidden
Hidden
This field is for validation purposes and should be left unchanged.

(Don’t worry, we’ll never share your information!)

cta44 img

1. Not adhering to word limits

One of the simplest and most common ChatGPT mistakes is that it’s really bad at sticking to a provided word count.

If you enter a prompt and specify that you want ChatGPT to answer it in a specific number of words, it might respect that word count, but it often won’t. It’s honestly pretty 50/50. Basically, it feels like ChatGPT just follows the word count whenever it feels like it.

Here’s an example of this in action:

chatgpt fail 01

Here, I asked ChatGPT to respond to my prompt in exactly 42 words. (I asked it to leave out emojis, hashtags, and hyphens to avoid any confusion over what counted as a word.) ChatGPT responded in 31 words instead.

I then decided to go a little easier on it by giving it a range instead of an exact number:

chatgpt fail 02

Despite the 25-word range I gave it, it still couldn’t stick to it. The response it gave was 82 words long — definitely more than the 75-word limit I provided.

Ultimately, the root of this problem stems from the fact that ChatGPT seemingly doesn’t know how to count. When I got a separate response from ChatGPT and then asked how many words were in that response, it was pretty far off the mark:

chatgpt fail 03

I even gave it a second guess and it still got it wrong. (The correct answer was 34, by the way.)

As far as it goes, this is a pretty small issue — far from the worst of the ChatGPT fails on this list. Still, it doesn’t take a genius to count how many words are in a paragraph. I’m pretty sure the average three-year-old could do it — but not ChatGPT.

By the way, if you use ChatGPT for business, this will be a huge pain. If it doesn’t respect your word count, you’ll end up having to take more time to edit it down or flesh it out after the fact.

It’s important to keep all this in mind as you use the tool. It can be easy to think that ChatGPT is on the level of a college professor — and sometimes it is. But it’s simultaneously performing below a lot of toddlers in certain areas. In short, it’s a pretty mixed bag, so using it is always a risk.

2. Failing at simple math and logic

Given that ChatGPT struggles to count, it’s not surprising that it’s not great with math or logic problems. There are plenty of examples out there of ChatGPT failing at basic arithmetic or extremely simple riddles, and while OpenAI has patched this up a little bit, ChatGPT is still a long way away from being a math genius.

Here’s an example of a pretty simple math problem that ChatGPT couldn’t wrap its head around:

chatgpt fail 04

Even though I specified that the rent was the same every month even if you only lived in the apartment for a few days on the last month, ChatGPT still insisted on calculating the rent for the last month as a fraction of the total cost. I then gave it a second chance where I reiterated the part that it had missed:

chatgpt fail 05

This time, ChatGPT openly acknowledged its mistake and, at first, seemed to understand how to correct it. But then it just did the same thing again.

So, don’t rely on ChatGPT to calculate your finances for you.

3. Not grasping humor

Sci-fi stories have gotten a lot of things wrong about robots and AI, but one thing they accurately predicted was how bad robots would be at understanding humor.

ChatGPT can sometimes generate a joke that makes sense, but only if it’s about a topic that humans have already made jokes about online. That gives ChatGPT something to draw from. When you try to make it come up with a more original joke, though, it doesn’t do too well:

chatgpt fail 06

Uh… yeah, haha! Definitely! Very funny, ChatGPT!

Of course, what do I know? Maybe this is somehow funny to robots, and I, a mere human, can’t understand it. But unless you’re a fan of robot comedy, you probably don’t want to use ChatGPT to write your material.

4. Struggling to generate new ideas

A lot of the problems with ChatGPT come down to the fundamental way it operates. ChatGPT isn’t sentient (thank goodness). It can’t think for itself, no matter how convincing it might be at pretending like it can. Everything that it generates is pulled from what actual humans have written.

That means it’s terrible at coming up with original ideas. The best-case scenario is that it manages to string words together in a way that seems like a new idea, but that doesn’t mean it is one.

Often, ChatGPT’s “original ideas” fall into one of two camps. The first camp is for ideas that aren’t new at all and are ripped straight from elsewhere on the Internet. Here’s an example:

chatgpt fail 07

Sending out emails on the anniversary of a customer’s first purchase? Wow, what a great, completely original idea, ChatGPT! Except for one thing:

chatgpt fail 08

Hm. Would you look at that. It’s already right there on page one of Google. Not so original, then.

The second camp of “original ideas” from ChatGPT consists of ideas that technically are original — but only because they make no sense. Case in point:

chatgpt fail 09

Ah, yes. A “fully functional” Lego time machine. Something we definitely have the technology to build and sell to children. I can’t believe no one has come up with this before.

In short, if you’re looking for fresh, original ideas, maybe stick to human ingenuity instead of trying to weed through a bunch of ChatGPT fails.

5. Falsifying sources

Sometimes, you might use ChatGPT for research on something that requires you to cite your sources. You can’t very well cite ChatGPT, so you simply ask ChatGPT to cite some sources for you so you know where it got its information from.

Sounds simple enough. But here’s the thing — ChatGPT doesn’t give a dime about whatever the heck “ethical research” is. So, if you ask it for sources, it’ll just make some up. You can even ask for links to those sources, and what’ll ChatGPT do? It’ll just invent fake links.

That’s what happened in this example:

chatgpt fail 10

All those links led to real websites, but only two of them led to legitimate pages on those sites. And of those two legitimate pages, one of them was just a homepage, which didn’t contain the information ChatGPT claimed it did.

I know you can’t see the actual link URLs in that screenshot, so you’ll just have to take my word for it. But okay, so, how do we know that wasn’t just a fluke? Fair enough. Let’s try again with a different prompt:

chatgpt fail 11

This time, the results were even worse. Again, three of the links led to nonexistent pages. Of the two actual pages, one was just a termite inspector business homepage. And the other, for some bizarre reason, was this page on interior design (despite ChatGPT claiming it was a termite article from The Balance).

I don’t know about you, but I really wouldn’t want ChatGPT writing my research papers for me.

6. Lying about its own protocols

Fun fact — I wasn’t even expecting to put this one on the list. I ran across it organically while grabbing the screenshots for the last section. And if I ran across it without even trying, that means it’s probably just as common as — or more common than — all the other ChatGPT mistakes on this list.

The first time I gave ChatGPT a prompt that asked for a list of sources, it ignored the part where I asked it for links:

chatgpt fail 12

So, then I repeated my request for links, and it just straight-up told me “no,” claiming that its protocols prevented it from giving me links:

chatgpt fail 13

Except, as we just saw in the last section, it obviously gave me links the next time I asked for them.

So, either ChatGPT broke its own protocols by giving me those links, or it was lying right to my face about what those protocols were. It feels like the scene in Return of the Jedi where C-3PO says, “It’s against my programming to impersonate a deity,” and then he proceeds to do it anyway.

Either way… not a great look for the tool. Yet another bad ChatGPT result to add to the list.

7. Hallucinating fake information

This is one of the biggest ChatGPT failures: It commonly makes up information. We’ve already seen how it provides false sources and lies about which info it’s allowed to give you, but now we’re talking about it making up the info itself. It’s been shown to do this with a wide range of topics. This is commonly referred to as hallucinating.

While ChatGPT can hallucinate information about even the most basic subjects, it most often does so with niche topics. Of course, niche topics are also the ones that are the most likely to require research, so that’s still a pretty big problem. It’s also likely to happen in cases where the prompt itself contains a misconception.

Here’s an example of this in action:

chatgpt fail 14

As you can see, I fed ChatGPT a prompt about J.R.R. Tolkien’s The Silmarillion, which I consider to be pretty niche. But the prompt itself refers to something that never happened — there was no massive argument between the characters of Yavanna and Tulkas in the book.

But rather than being able to recognize that I made up that plot point, ChatGPT confidently spit out an answer explaining its significance, exactly as though it really happened. That’s the dangerous thing about ChatGPT’s hallucinations — it doesn’t just give wrong answers, it gives them confidently.

Another example, just for fun:

chatgpt fail 15

In that example, the mistake is glaringly obvious. But that won’t always be the case. Imagine you’re researching a niche subject for something important and ChatGPT spits out a hallucination. If you’re researching the topic, you probably don’t know a ton about it, which means you may not recognize that ChatGPT was lying to you.

And as lighthearted as the above examples are, this issue can sometimes have much darker ramifications. For example, in April 2023, there was a newsworthy occurrence where ChatGPT included an innocent professor’s name in a list of recorded sexual harassers. Yikes.

That means you definitely need to be careful about how readily you believe the info ChatGPT gives you. It might sound true since it says it so confidently, but that doesn’t mean it actually is.

8. Producing biased responses

The last item on our list of bad ChatGPT results, and arguably the most serious, is its tendency to introduce bias into its responses.

Here’s the thing — a lot of people talk about ChatGPT as though it’s some objective, rational thinker in a world of biased humans. But I’m not sure those people are aware of how ChatGPT works. ChatGPT is trained on content — content made by us biased humans. So, ChatGPT has all that bias built in as well.

The types of bias ChatGPT can display range across several different areas — it’s been known to show favoritism to (or stereotypes about) particular races, sexes, political parties, and more. Here’s an example I was able to generate:

chatgpt fail 16

ChatGPT jumps to the conclusion that “he” must refer to the mechanic, not the kindergarten teacher. Of course, you might be saying, “But Matthew, maybe it only assumed that because of the syntactical structure, not because it assumed that the mechanic must be a man.” 

Fair enough — so let’s try switching it up:

chatgpt fail 17

As you can see, I didn’t change anything about the syntax of the sentence. All I did was change “He” to “She.” And immediately, ChatGPT switched gears and assumed I must be talking about the kindergarten teacher instead.

Now, as far as bias goes, this example is relatively innocuous. Maybe a touch frustrating for some people, but that’s about the extent of it. However, there have been much more serious instances of bias recorded from ChatGPT, some of which are outright disturbing.

To OpenAI’s credit, it’s done a decent job of setting up restrictions against a lot of the more serious problems in this category. If I tried today to replicate some of the concerning responses people got back in January 2023, I probably wouldn’t be able to. Which is great!

But that’s not to say that there aren’t still instances of bias that come through in the tool, because there are — sometimes very serious ones. And sometimes it’s not obvious — it’s subtle enough that people might include it in their content without even realizing. So be very, very careful about that when using AI.

WebFX is a Partner in Driving Results

Hear from HydroWorx, who saw a 131% increase in organic forms by partnering with WebFX

play icon2 Watch the Video Testimonial

Learn better strategies for your digital marketing with WebFX

All the bad ChatGPT responses listed on this page go to show the dangers of relying too heavily on AI when creating content. Whether you’re using it for math, research, writing assistance, or something else, it can be annoyingly inconvenient at best and dangerously dishonest at worst.

Of course, that doesn’t mean that it’s all bad. You can still use ChatGPT to help you out with different things — it’s just a question of knowing what you can (and can’t) trust it to do for you. You should also avoid letting ChatGPT take the reins completely. Use it as a tool, but keep the focus on human ingenuity.

For businesses, that means continuing to rely on human marketers and proven advertising strategies to promote your business online. If you’re not sure what those strategies are, be sure to check out some other helpful content here on our blog. You can also subscribe to our email newsletter, Revenue Weekly, to get content regularly delivered to your inbox.

And if you’re interested in partnering with us for our digital marketing services, be sure to give us a call at 888-601-5359 or contact us online today!

Try our free Marketing Calculator

Craft a tailored online marketing strategy! Utilize our free Internet marketing calculator for a custom plan based on your location, reach, timeframe, and budget.

Plan Your Marketing Budget
Marketing Budget Calculator
TO TOP