richmondez

There is a real risk that will become the attitude though, no need for security researchers, the LLMs got us covered. Great if true but it’s probably not.

They are front ends for an LLM service, no LLM, no more agent activity. Just like a bot net if the command and control goes down. Would be easy to just change whatever api they are using or change some urls surely.

Who in their right mind is exposing them directly to the internet? Poorly configured routers to blame here I assume?

Can’t most serious compilers produce reproducible builds these days given the same build environment. I know there has been a drive towards reproducible builds in general for security verification purposes.

You should have stopped at “don’t share on social media”.

There are normally plenty of examples when an effective monopoly is broken of prices coming down to stay competitive, patents expiring on drugs normally sees prices tumble. A single example where a company becoming an effective monopoly lowered prices might be a better comparison because I’m fairly sure that hasn’t happened.

What I’m saying is the way it’s marketed to be used is much closer to the wrong way and it’s far less of a productivity enhancer when used how it really should be and relies on having a lot more professional level IT skills.

Local stuff absolutely can do damage to your local data, unless your backups are pull rather than push or are immutable on a file system they are potentially vulnerable. Unless the operating system specifically prevents the agent process from doing something telling it not to is like asking it to pinky promise not to, nothing stops it if the LLM output it relies on says to do so. It’s not deterministic like a regular program because it relies on the output from a chaotic black box text generator. All you can say is that it probably won’t do those things if it says it won’t.

All purchased download music is pretty much drm free, wish other forms of media were without having to sail the high seas.

So basically use it as a toy for entertainment purposes only… Except it’s not marketed to be used that way and far to many people aren’t smart or educated in computing enough to use it that way. Accepting a bash command you don’t understand is the same as running a bash command you found on the Internet… You shouldnt, but people are far more trusting of the stuff an LLM powered agent comes up with.

This is exactly right but also removes all the “productivity benefits” of having an LLM driven agent do all the things for you that marketing says you should use it for. You are supposed to push the “do whatever the fuck you like to get the job done” button so it can burn tokens without any oversight.

Disagree, the agentic LLM use does not have any real agency of its own, the user giving it access and a request and not preventing the possible illegal ways it could fulfill that request and entirely on the user for being ignorant and irresponsible trying to shift work onto a system with chaotic behavior.

Maybe but even there there are limits on how much they will be willing to spend for junior level programming no matter how much of a ship fast fix broken methodology they adopt.

The public assumption that AI can’t tell fact from fiction and confidently makes stuff up doesn’t seem at odds with them being able to spin a good yarn.