What LLMs Can't Do (And Why That Matters)

Shawn West
Oct 27, 2025
7 min read

Updated: Aug 10

An AI chatbot will hand you a wrong answer in the same calm, fluent voice it uses for a right one. It never sounds unsure. This is a plain-English tour of the four things large language models genuinely can't do — and, more usefully, why those limits exist, so you can spot trouble before it reaches someone who trusts you.

Priya had been at the nonprofit for two weeks when her manager asked for a one-page summary to put in front of the board. Nothing fancy — a paragraph on why their tutoring program mattered, one supporting statistic, and a quick line on the budget. She opened a chatbot and typed a normal request: Give me a statistic about after-school tutoring and reading outcomes, with the source, and add up these four program costs.

Back came a clean paragraph. "A 2019 study by the Hastings Education Institute found that students in structured after-school tutoring gained an average of 2.3 grade levels in reading." The four costs were summed to a tidy total. It read like something a competent colleague wrote. Priya nearly pasted it straight into the deck.

Two things in that answer were wrong. The study didn't exist — not the institute, not the number, not the 2019. And the budget total was off by a few hundred dollars. Neither mistake announced itself. That's the whole problem with large language models, and it's worth understanding why it happens, because once you see the machinery, the limits stop being surprising and start being predictable — which is exactly what makes them manageable.

If you want the friendly version of how these models work in the first place, What Are Large Language Models? is the place to start. Here we're doing the other half: where they break, and why.

The one thing to understand: it's predicting words, not knowing things

Everything below comes from a single fact about how an LLM works. When you type a question, the model isn't looking anything up. It's doing something closer to the world's most sophisticated autocomplete: given all the words so far, it predicts the next most likely word, then the next, then the next. That's it. It was trained by reading an enormous amount of text and getting very, very good at guessing what word tends to come next.

This is genuinely powerful. "Likely next word," repeated billions of times over, produces fluent essays, working code, and helpful explanations. But notice what it does not include: there is no step where the model checks whether what it's saying is true. It's optimizing for plausible, not for correct. Most of the time plausible and correct overlap, which is why the thing is useful at all. The trouble starts exactly where they come apart — and the model can't tell the difference, because it was never measuring truth in the first place.

Hold that idea and each limit below becomes obvious rather than mysterious.

It makes things up — and sounds certain doing it

The invented study Priya almost cited has a name: a hallucination. It sounds like a glitch. It isn't. It's the autocomplete doing precisely what it does.

Ask for "a statistic with a source," and the model produces the words that most plausibly follow that request: a number, a study-shaped name, a year. Real citations look like "a 2019 study by the [Institute]," so the model generates text in that exact shape. It has no database of studies to check against and no little voice that says I'm not sure this is real. A true citation and a fabricated one are produced by the identical process, which is why they arrive with the identical confidence.

That confidence is the trap. Bad information that reads badly gets caught. Bad information that reads like a polished sentence from a smart colleague gets pasted into a board deck. The fluency isn't a sign the model knows what it's talking about — it's just the model being good at its actual job, which is sounding right.

The catch: treat any specific, checkable fact from an LLM — a statistic, a quote, a citation, a court case, a product spec — as unverified until you've seen it somewhere else. Priya's move is a ten-second web search for "Hastings Education Institute." Nothing comes up. That's the answer.

It can't do the math

The budget total was wrong for the same underlying reason. The model didn't add the four numbers. It predicted what the sum "looked like" it should be, based on patterns in text — and prediction is not calculation.

LLMs handle arithmetic by pattern-matching, not by computing. They've seen "2 + 2 = 4" enough times to reproduce it, so simple sums usually come out right. But ask for something it hasn't effectively memorized — four real dollar figures added together, or two four-digit numbers multiplied — and you get a confident answer that's frequently a little off. "A little off" is the dangerous kind, because it's plausible enough to survive a glance.

The catch: never let the base model be your calculator. Do the sum yourself, or use an actual calculator or spreadsheet. (Some AI tools now connect the model to a real calculator or code, which fixes this — but you have to know whether the tool you're using does that, and Priya's chatbot didn't.)

Its knowledge stops at a date — and it won't warn you

Suppose the tutoring statistic had been real, but from a rule or figure that changed last month. The model still might not know. An LLM's knowledge is frozen at a training cutoff — the point when its reading stopped. It doesn't know today's news, this week's prices, whether a company still exists, or that a policy was updated after that date. Unless the tool is specifically wired to search the live web, it's answering from a snapshot of the past.

And here's the part that catches people: it usually won't tell you it's out of date. Asked about something recent, it will often produce a confident answer built from older patterns rather than saying "I don't know." Same mechanism again — plausible beats accurate.

The catch: for anything time-sensitive — laws, prices, current events, "the latest version of X" — assume the model may be stale, and confirm against a live source. If the tool offers a web-search or "browse" mode, that's when to use it.

It imitates reasoning — it doesn't actually reason

The subtlest limit is the one that fooled Priya's instinct to trust the whole thing. When an LLM lays out an argument step by step, it looks like it's thinking. What's really happening is that it learned the shape of reasoning from millions of examples and reproduces that shape. The steps look logical. The conclusion can still be wrong — and because the model isn't checking its own logic against reality, a coherent-looking argument with a broken middle sails right through.

This is why it can write a paragraph explaining why an approach is sound and be completely mistaken, in fluent, well-organized prose. It doesn't understand the tutoring program, the budget, or the board. It understands what sentences about those things tend to look like. For anything where being wrong has real consequences — money, health, legal, safety — that gap between "sounds like it thought it through" and "actually correct" is exactly where the damage lives.

(Developed example — composite scenario. Priya, the study, and the budget are illustrative, not a real case.)

Why this matters more than it seems

Priya caught it because she paused on one instinct: before this goes in front of the board, does the source actually exist? That single question exposed both errors — the fake study and, when she re-checked the numbers, the bad sum. The version of this story that goes wrong is the one where she doesn't pause, the board cites a study that never existed, and someone eventually asks for it.

None of these limits mean LLMs are bad or that you shouldn't use them. They're extraordinary for first drafts, explanations, brainstorming, rewording, and getting unstuck — anywhere a human is going to look at the result before it counts. The failures happen when people treat "sounds authoritative" as "is verified," and skip the look. If you want to get better results out of the model in the first place, Prompting 101: How to Talk to AI helps — but better prompting reduces mistakes, it doesn't remove the need to check. And if you're weighing which tool to trust with what, Local vs. Cloud LLMs walks through those tradeoffs.

Here's a quick map of the four limits, the reason behind each, and your move:

The model can't…	Because…	Your check
Guarantee facts	It predicts plausible words, with no truth-check step	Verify any specific fact against another source
Do reliable math	It pattern-matches sums instead of calculating	Do the math yourself or use a calculator
Know recent things	Its knowledge is frozen at a training cutoff	Confirm anything time-sensitive against a live source
Truly reason	It copies the shape of logic, not the logic	Check the conclusion, not just that it sounds right

The habit worth keeping

You don't need to memorize four rules. You need one habit, and it fits in a sentence: before an AI's answer leaves your hands and reaches someone who's counting on it, find the specific claims and check them yourself.

Practically, that's three quick passes on anything that matters. Any fact — search for it and confirm it's real. Any number — redo the calculation. Anything recent — check the date against a live source. If the answer is just a rough draft you'll rewrite anyway, relax; the model is great at that. The check is for the moment the output stops being a draft and starts being something you're vouching for.

That's the whole discipline. The model is a fast, fluent, tireless assistant that will occasionally state something false with total confidence and no idea it's doing so. Once you expect that — once "it sounds sure" stops meaning "it's right" — you get almost all of the upside and very little of the risk. Priya kept her habit for one board deck. It's the same habit whether the stakes are a paragraph or a decision that costs real money.