a1studmuffin

And we’re about to watch Anthropic go through the same thing with their IPO. Late-stage capitalism sucks.

Read my post above to a different commenter, there’s a new type of ASIC being talked about that’s very different to an NPU or GPU, no RAM necessary.

LLM ASICs are different to NPUs. The idea is to encode a specific model’s weights (all billions of them) as a gigantic resistor network on the chip itself. Analog, where ohms law becomes the processor, and no RAM is necessary - the chip IS the model. It would pave the way for cost effective, fast, low-power local inference. Given the analog circuitry, there would be issues like interference, temperature sensitivity and so on, but we’re seeing models get good results even when heavily quantised, so the idea has legs. The big disadvantage is you’re stuck with that one model forever, so we’re likely to see these first appear in non-consumer applications like automotive, defence etc. A pretty exciting concept that solves some of the environmental issues around AI.

More info for the curious: https://taalas.com/the-path-to-ubiquitous-ai/

I’m really interested to see what comes of the LLM ASIC idea - it worked wonders for hardware video codecs. I could see the smartphone industry picking up again on annual releases worthy of upgrades if they included dedicated AI chips not beholden to a cloud subscription. I think we’re reaching the point that the frontier models are becoming “good enough” to use for a while before uploading to the next.

I swear this is the real reason AI companies are concerned about watermarking - not for the public good, but so they can keep their model training inputs clean.

Reminds me of Sam Altman saying he uses AI to make a podcast about his kids sports game so they can listen to it on the way to school. As if talking to your children wasn’t an option.

All these tech bros creating solutions in search of a problem.

And even that would be considered absurd by yesterday’s standards. The entire game ROM for Super Mario Bros on NES is 40KB.