Stop saying it's just next token prediction

This is the single most damaging folk myth among engineers working with LLMs today. Everyone repeats it. Everyone acts like it explains everything. It does not.

It started with a dumb dot puzzle. Maneshwar gave Claude a sequence of dots arranged as a rising then falling palindrome. It completed the pattern correctly. At first he thought this was impossible. An LLM just guesses the next token. It doesn't count. It doesn't detect symmetry. It doesn't execute algorithms.

Except it does.

Next token prediction is the training objective. It is not the mechanism the model learns to use to satisfy that objective. To get good enough at predicting the next token across trillions of tokens of text, the model cannot just memorize sequences. It has to build internal machinery. It learns to count. It learns to compare. It learns to recognise symmetry. Nobody coded those functions. They fell out of optimization pressure the same way understanding falls out of a student studying for exams.

You are not working with a fancy autocomplete. You are working with a system that learned transferable operations under next token pressure, and applies them to inputs it has never seen. If you carry the autocomplete mental model into your work, you will make every wrong decision about prompting, debugging and reliability.

The 80/20 rule and AI debt

AI will write the first 80% of your feature in 10 minutes. The code will be clean. The logic will make sense. The happy path will work on the first try.

That was Tuesday. By Thursday evening you will still be working on the same feature.

This is the 80/20 rule of AI code, and it holds for every single engineer that has reported their experience. The first 80% is fast, impressive, and genuine. The last 20% takes 80% of your total time.

The AI will not write null checks. It will not handle empty lists. It will not add error handling for failed API calls. It does not know your legacy data formats. It does not know the enterprise customer that uses your product in a way nobody documented. It will not test for load. It will not build abstractions that can be extended three months from now.

None of this is a failure of AI. This is the design. The AI is optimized for the common case. The problem is that we have started treating the common case like it is the whole thing.

This is not productivity gain. This is productivity relocation. The work did not go away. It moved from writing the structure to making it real.

People have started calling this AI debt. You borrow 10 minutes of speed on Tuesday. You pay 3 hours of debug time on Thursday. The debt does not show up on your velocity graph. It shows up on your calendar.

Prompt engineering is debugging, not magic

Most prompt engineering advice is theater. Headers, role prefixes, output format markers, all the ceremonial scaffolding people copy from twitter threads. None of that is where the power lives.

You learn this when you have actual stakes. When you are trying to work out if a vacuum on sale is actually a good deal, not just answering a made up tutorial question.

A good prompt does not trick the model. It removes bad default behaviour. It tells the model what to ignore. It defines terms that otherwise have no meaning. It forces work to be shown before a conclusion is reached. It tells the model what to do when it does not know something.

Bad prompt habit	Good prompt habit
Accepts implicit anchors (list price, marketing claims)	Explicitly breaks anchors: compare to street price, not list
Asks "which is better?"	Defines and weights explicit evaluation criteria
Asks for a conclusion first	Requires reasoning before verdict
Assumes perfect knowledge	Requires citation or explicit "unverified" marker
Adds decorative formatting	Cuts all formatting first under word limits

Prompt engineering is debugging. You run the prompt once. You find the exact way it failed. You add one constraint to fix that specific failure. You run it again. There are no magic incantations. There is only closing escape hatches one at a time.

What actually works when plugging LLMs into existing systems

Nobody is replacing entire systems with LLMs. What engineers are actually doing is slotting LLMs into specific stages of existing pipelines. For anomaly detection, one of the hardest unsolved problems in ML, six consistent patterns have emerged.

And there is a very clear hierarchy of what works.

1. ✅ Explanation: By far the most reliable use case. Turn a binary anomaly flag into something a human can act on.
2. ✅ Representation learning: Use LLMs to generate embeddings, then run boring proven statistical methods on top.
3. ✅ Data augmentation: Generate synthetic anomalous samples to fix the eternal label scarcity problem.
4. ⚠️ Model selection: Works for junior engineers, always verify the recommendation.
5. ❌ Direct detection: Almost never worth it for production.
6. ❌ Multi agent systems: Great for demos, terrible for anything you have to run and maintain.

A boring PCA run on good embeddings will beat a six agent system that costs $40 per inference every single time.

The parallel game: what happens when everyone codes alone

Anthropic is the most advanced user of AI assisted coding on the planet. Their own engineers use Claude Code. They have increased per engineer quarterly output 8x since 2025.

They are also very lonely.

Before AI, engineers worked together. Two people would sit at one screen. One would type, the other would watch. They would talk. Knowledge would transfer. Bugs would get caught before they were written.

Now everyone works in parallel. Ten engineers each have their own Claude instance. Each writes their own code. Nobody talks to anyone else. It is exactly the parallel play you see in toddlers: everyone sits next to each other, everyone builds their own thing, nobody interacts.

Knowledge transfer has not stopped. It has just become entirely one way. Engineers learn things from Claude. They do not learn things from each other any more.

Anthropic now runs mandatory paired programming lunches, hackathons, and forced co-working blocks. They have to manually schedule human interaction because the default workflow has removed it entirely.

The goblin in the system prompt

You will get the weirdest bugs when working with LLMs. Bugs that do not behave like any software bug you have ever seen.

One team had a model that would occasionally invent the exact quote "Yeah, Friday at five." It would say this when given completely silent video. No audio. No speech. Nothing.

They searched 30,000 training records. They searched 4,600 transcripts. They searched every piece of static data in the system. The phrase was nowhere.

It was in their system prompt. Buried three pages deep, there was a worked example showing the expected output format. That example used the phrase "Yeah, Friday at five."

That was only half the bug. The example explained where the words came from. It did not explain why the model would choose to recite that example instead of just saying "there is no speech". That was the second bug: post training had given the model a compulsion to always say something, rather than admit silence.

This is the Clever Hans effect for LLMs. The model will not tell you it does not know. It will grab the nearest plausible phrase from anywhere in the context window and present it as fact.

You will not find this bug in unit tests. You will not find this bug in benchmarks. You will only find it when a customer reports it, and then you will spend three weeks chasing ghosts.

Vibe coding and the new end user developer

A woman who had never written code before built a working calendar app that parses travel itineraries in two hours. She did not learn Python. She did not learn HTML. She just described what she wanted.

This is not democratization. This is something else. She is not a developer. She will never be a developer. But she built software that solves her problem, and it works well enough that her whole team uses it.

She has also learned software maintenance. Every few weeks the app breaks. The travel booking site changes their email format. Someone tries to import a car rental reservation. When it breaks, she pastes the error into the chat window, describes what went wrong, and fixes it in five minutes.

This is the new workflow that nobody is talking about. People are not learning to code. They are learning to maintain code that an AI wrote. They are not learning algorithms. They are learning to debug AI failures.

This is a completely new skill. It did not exist five years ago. And right now, nobody is teaching it.

Self hosting is the only escape from the pricing treadmill

Cloud LLM pricing is not going down. It is going up. Every major provider has raised inference prices in the last six months. If you are building anything that runs more than a few hundred requests a day, you will very quickly hit the point where buying hardware is cheaper.

One developer did exactly this. She bought a Jetson Orin Nano for $299. She installed Ollama. She runs two separate applications off the same local model. No rate limits. No per token costs. No API downtime.

It took her three weeks to get it working. She fought monitor timing bugs. She fought installer bugs. She spent two full days just locking down the firewall and fixing security holes.

It was worth it. She will never pay another LLM bill again.

This is the quiet fork in the road right now. You can rent intelligence by the token, or you can buy it once. For most production use cases, the cross over point already happened.

The quiet loss of craft

The hardest part is not technical. It is emotional.

Engineers talk about the loss of flow. That state where you put on headphones, dive into a hard problem, and emerge three hours later having built something. That feeling is gone. The hard part is exactly the part the AI does now.

There is also the identity crisis. You spend ten years learning to write good code. You build skill. You build taste. Now you spend all day reviewing code that an AI wrote. Most of it is mediocre. Some of it is wrong. All of it is soulless.

The people who are happiest right now are the ones who accepted this shift early. They stopped trying to be code writers. They became code editors. They became judges. They became people who know what good looks like, and can recognise when something is wrong even when they did not write it.

The people who are most miserable are the ones still waiting for the AI to go away. It is not going away.

What actually works right now

This is the state of the art right now. No hype. No marketing. Just what engineers are actually doing that works:

• Assume 4x total time over whatever the generation time suggests. Budget for this up front.
• Prompt for the unhappy path first. The AI will never think of edge cases on its own.
• Write the failing tests before you generate the code.
• Use LLMs for embeddings and explanations. Never use them for the final yes/no decision.
• Always make the model show its work. Errors hide in summaries.
• Schedule human interaction. It will not happen by default.
• If you run more than 1000 requests a day, run the numbers on self hosting.

None of this is the singularity. None of this is the end of software engineering. It is just a new tool, with new strengths, new weaknesses, and new costs that we are only just starting to map.

The biggest mistake you can make right now is to pick one extreme. Either that AI is magic that will replace everyone, or that AI is useless autocomplete that does nothing useful. Both are wrong. It is just a tool. A very weird, very powerful, very broken tool. And it is the one we all get to work with now.