It’s already happening. Software instability is becoming a major problem.

Honest answer? Not much… yet. As the stupidity of “tokenmaxxing” and replacing developers with hallucinations-and-plagiarism engines really comes into focus, I expect developer hiring to ramp back up.

Meanwhile, in the real world, my company has locked down usage of Anthropic’s top-tier models, because costs were out of control. Even the low-end models are usage-capped, and developers are starting to write code by hand again.

And we’re busily setting up local models we control ourselves on our network edge. We’ll probably end up with a modest budget for frontier models (until the model providers collapse), but most of the coding will be done by local models and developers themselves.

And that seems to be the way the entire industry is going, unless Anthropic starts subsidizing tokens with investor money again.

In conclusion: Get fucked, Dario.

As I’ve mentioned elsewhere, not if by “information” you mean semantic content that a mind can process. What they have are vector fields (essentially just numbers) with statistically more or less likely relationships.

If I say, “take me out to the ballgame” to an LLM, the tokens representing the words in the next verse of the song are statistically “close” in the vector database, so it’s likely to generate them. But that doesn’t mean it actually knows the lyrics… or even has those lyrics recorded in a regular database anywhere.

That’s why they hallucinate. The model determines that the next token is something nonsensical, but it has no way of understanding that it has made a mistake. In a sense, it actually hasn’t made a mistake. It’s done exactly what it’s designed to do. It’s just that in the case of hallucinations, its output isn’t useful.

They’re information, but not the same information that was used to create them.

No, they really don’t. That’s not how they work. At least, not if the “information” you’re talking about is real semantic content that real minds can process.

Every piece of information you think an LLM has access to is actually just converted into a stream of additional tokens that are fed into the model to (hopefully usefully) modify the next tokens it predicts. That’s not the same thing as having actual access to information. Tokens are just numbers with statistically more (or less) likely relationships to each other.

I’m not trying to downplay LLMs. They’re architecturally interesting and have genuine uses. I’m just trying to head off a bit of technical inaccuracy.

The important thing to remember is that it actually has zero access to information, because that’s not how LLMs work.

At their core, they’re vector databases, and they’re trying to probabilistically come up with the next most likely token in a stream of tokens found in the DB. You can manipulate the stream by injecting text such as the content of existing files (which becomes more tokens) into the stream, but it never actually understands any of it.

That’s why hallucinations are inherently unavoidable. It’s really all just hallucinations. It’s just that you can sometimes get useful text from their hallucinations if they happen to comport with reality.

Jesus Tapdancing Christ…

Isn’t this the breach that was widely reported on in March? And they’re just now disclosing the scale of the exposure?

We’re sad because a lot of people have bought into overly-hyped bullshit factories made by assholes who claim they can replace us.

They can’t, but our employers are currently too uninformed (or addicted to the bullshit, or desperate to prop up their stock prices) to realize that the bullshit factories aren’t capable of replacing us.

That’s causing a lot of turmoil in the form of unnecessary layoffs and rehires, worsening software quality, and general job insecurity.

What we want is software that behaves predictably. Since LLMs don’t do that, we don’t want them or their “agents.”

For fuck’s sake.

It’s listed as medium severity and appears to require the hacker to already have terminal access to the system. It’s also already patched and there’s a quick and easy workaround if your distro doesn’t have the fix yet.