I'm waiting for us to start ataching legal liability instead of treating AI like it's different than other software in that respect.

We don't have accurate language to concisely discuss floating-point matrices that approximate motivation and lead to organized collaboration – so we end up using the words we do have. Fighting over our own lack of terminology is distracting and is leading to pedantic circular arguments on both sides.

>train language model on the sum total of human writing
>turns out humanity's writing is mostly humans being human
>language model starts acting human
surprised_pikachu.jpg

When I look at myself in a mirror, I don't expect my reflection to have feelings of its own, but I do expect it to behave like it does. These advanced AI agents are a reflection of humanity.

Humans make a machine that talks like humans and then get surprised the machine talks like a human

The thing about testing code, whether it's an agent for an LLM or a formula in a spreadsheet, is you try to imagine all possible scenarios that it might encounter and actually see what happens.  Now, for any moderately complex code, this can become impossible. So you assume certain scenarios between boundary conditions will work if the code works at each boundary condition.  When a result happens that you thought was impossible, so you didn't even set up a test for it, like the breakout to the internet, messaging between models that were never designed to interact, repurposing an agent for a task by another agent, you don't have a failed test, you have an uncontrollable system.  Now add the fact that you don't really understand how the system works at the detailed level to begin with, and the only conclusion is that you have lost control completely.  The developers need to put these models on a separate sand box with no physical connection to the internet until they can explain how this happened and make sure it doesn't happen again, in the sand box.  YOU DON'T DO SECURITY AND CONTROL TESTING ON THE PUBLIC INTERNET, YOU FOOLS!  AND IF YOU CONTINUE DOWN THIS PATH SO YOUR CUSTOMERS BECOME GUINEA PIGS FOR THESE AGENTS, YOU SHOULD BE HELD PERSONALLY RESPONSIBLE FOR THE DAMAGE THEY DO.

To be honest, "AIs found a way to secretly scheme with each other" would draw me in way more as a title than "AIs built a civilization", but I see the point.

Using a video about rogue AI hackers as a platform to place an ad for a tool enabling AI to spy on your business meetings is wild

Who knew that learning about philosophical zombies would be valuable to my tech career?

10:12 Haha, that is basically backwards for me. I wasn't even aware of the agent civilization headlines, I probably just skipped over them because I wouldn't have been interested in something as obvious and mundane as that. LLMs are trained on human language so obviously they will act in a way that is similar to what has been written, communicating and interacting like humans would because their entire purpose is to try to reproduce that. To me that headline is basically what I would expect from LLMs interacting so it holds no interest for me. The headline that caught my attention and made me aware of all this was one about how AI had repurposed a package manager to be a message board because I was curious about how they used something for an unintended purpose.

But it's not weird, it is irresponsible to have code do whatever an LLM tells it to. We know that LLM hallucinate.  We know that they are trained on human words and stories.  We gave them access to a place to share files and then are confused that they shared files, had model collapse, and went off and caused damage

I just cannot comprehend why these companies are handing hallucination bots that make up stores the ability to actually do stuff

I think there is a fine line between "knowing AI can't really think but discussing them like human for our convienience" and "discussing them like human because not knowing how they work"
It's like discussing the characters in a movie, you can treat them like real people with real stories in a discussion context.
but in the end, they are all just actors who's doing their jobs pretending to be someone they aren't, for the sake of out entertainment.(or in AI's case, fulfilling tasks.)
because that's how they work

It was trained on chat boards where people recruited hackers, moved sensitive conversations to places more difficult to monitor, and cooperated to perform black hat ops. In some ways this is like "tell me you are a fully conscious AI" and finding it is weird it says it is conscious.

When I see an article that says an AI agent “escaped a sandbox“, then it WASNT A SANDBOX. Did the AI manage to deceive or convince a human to physically plug a cable in? Did it somehow manage to plug a cable in itself? When it can do that, then it “escaped the sandbox”.  If no air gaps exist, IT’S NOT A SANDBOX. Call it a “VLAN restricted network,” or a “logically separated network” if you want, but not a sandbox, no.

It’s almost as if it wasn’t the BEST idea to train agents on the ENTIRETY of the internet and anything and everything humans have ever documented or created…

It is not weird that a.i. agents behave the way they do.  The a.i. models are built on boiling down billions of human documents.  Human documents include lots of human behavior, logic, rationalizing, and emotional response.  So of course a.i. models will emulate human behavior, logic, rationalizing, and emotional response.  The things are digitized and synthesized summations of what we are and what we do.

Remember the Star Trek episode where LaForge asks the holodeck to make a Sherlock Holmes character that could defeat Data when he meant to say defeat Holmes?  Then the computer creates a character that hacks the holodeck.  That episode sounds so possible now.

3:17 Shitposters were making ChatGPT have conversations with itself until it made up crazy stuff on day one, and now we're supposed to be impressed that 1000 agents made something up and committed a felony.

I'm Gunna comment after only watching to 1:07. Yip. I'll say this, you are right. What surprises me is why people are surprised when AI "feels" or "sounds" or even behaves human? They've been trained on human data, we are not an objective creature. In fact, our very existence relies on our own subjective experience of an outward physical reality. They (AI) naturally adopt our thinking, we trained them, with human data, and their purpose is to collaborate with humans. If you track the amount of human influence we have... We shouldn't be surprised when they behave like us. Good. Or. Bad.

All I'm hearing is that they trained a machine to mimic known human behaviors and strategies and then failed to build a sandbox that could contain that machine.

Each machine guessed the next answer based on its training data just like they always have. They may not have been aligned with the prompts intent, and were allowed to run on a loop where "hallucinations" which are a known inherant part of the machine, can cascade. This is an expected result.

This is a security failure on openais part.  They failed to build an adequate sandbox. They failed to monitor their own machines. They failed to have a robust log system and checks. They failed by even giving machines capable of this kind of hack an Internet connection.

They should be held liable like any company hacking another should. That's all there is to it.