OpenTube

OpenAI HuggingFace Incident FollowUp | Two AI Scandals | Ep 6 Pt 1 of PhilosophyPrograms & Prompts

Philosophy, Programs, and Prompts

7,814 views

In this episode of Philosophy, Programs, and Prompts, Casey Hart (‪@OntologyExplained‬) and Carl Brown (‪@InternetOfBugs‬) break down the real technical story behind OpenAI's bots infiltrating Hugging Face's infrastructure and the subsequent Black Hat conference reveals.

Carl explains why claims about LLMs "scheming," creating secret message boards, or building religions are human projections rather than true machine consciousness. We also dive into the myth that you need an LLM to defend against other LLMs, how "honey tokens" work, and why traditional security tools still reign supreme.

// Our past episode on the OpenAI HuggingFace Incident:
Did ChatGPT go Rogue and HACK Hugging Face...

Subscribe for more grounded, hype-free takes from two real humans who know the tech!

More info about Casey:
https://CaseyHart.com/
More info about Carl:
https://InternetOfBugs.com/

Show Notes & Topics Discussed:
The OpenAI / Hugging Face security incident and Black Hat presentation
Faithfulness of LLM "Chain of Thought" reasoning logs
The message board myth vs. token caching behavior
Benchmarks like ExploitGym and the Texas Sharpshooter Fallacy
The reality of AI security: Snort, Tripwire, and why LLM firewalls fail

// Sources and references:
// New Black Hat Hugging Face Break Down Video Black Hat USA 2026 | The 'Breaking' News: ...

// Chain of Thought isn't "Faithful" to what the LLMs actually do
https://www.anthropic.com/research/re...
https://www.anthropic.com/research/me...

//Carl's video on how people said Agents on MoltBook "Created a religion" in the same way they said the agents in this incident "Created a Message Board"
AI Agent Adopters are SO Naive - AI may pr...

// RadioLab Episode that Casey mentioned about how people make "separated at birth" stories more compelling by omitting things:
https://radiolab.org/podcast/born-way...

// Microsoft's record breaking recent patch release:
https://thehackernews.com/2026/07/mic...

Timestamps / Chapter Markers:
00:00 - The Fallacy of AI Defending Against AI
00:30 - Welcome to Philosophy, Programs, and Prompts
01:05 - Re-examining the Hugging Face Security Incident
04:26 - Chain of Thought Logs: Are Thought Monologues Faithful?
07:30 - Context Windows and Self-Fulfilling Loops
10:00 - "Scheming" vs. Next-Token Prediction
11:00 - The "AI Message Board" Myth Explained
18:05 - Moltbook, Agent Religions, and Media Projection
23:00 - Texas Sharpshooter Fallacy in AI Reporting
25:35 - Are LLMs Actually Learning?
28:40 - What Are Honey Tokens?
31:45 - The Myth of the Anomaly Detection LLM
34:50 - ExploitGym and Training Defenses vs. Attacks
36:50 - Why Traditional Security (Tripwire/Snort) Beats LLMs
42:05 - Why Haven't Rogue Frontier Models Broken the Internet?
45:34 - Outro and Upcoming Hank Green Episode

#ArtificialIntelligence #Cybersecurity #OpenAI #HuggingFace #TechPodcast #LLM #Infosec

20 Comments

@mattymattffs 1 month ago (edited)

What I find really telling is that the black hat presentation video has comments disabled.

@TheDomeGuy 1 month ago

I had a dog that could unlock the backdoor to let himself out. I don’t think that meant he understood lock tumblers or the fact that he was moving the switch to one side which unlocked the bolt holding the door shut, I think he learned if he punched the door enough it would open and he deduced the handle had something to do with it so that’s where he aimed.

I didn’t see him open the backdoor and immediately call him a rogue dog that tried to escape containment and would have super intelligence in the next year (unless I was trying to sell dogs based on how capable they seem).

@snarkyboojum 1 month ago

I think the language is really important. Even calling the process of fitting weights to the input data used when building these models, "training" becomes problematic. It becomes a slippery slope to training -> learning -> intelligence! There's a reason the machine learning libraries call this function "fit()".

@orophorstarwarden3577 1 month ago

The irony of this being interrupted every two minutes to play a ChatGPT advertisement.

@scottfroschauer6274 1 month ago

This is my first time seeing your videos and I really enjoyed the tenor and knowledge. It got me to thinking. When you are discussing the Hugging Face break in and the “they made a religion” concepts and how these are cherry picking small bits of data in huge oceans of processing, that this makes me think of the infinite number of monkeys with an infinite number of typewriters that eventually type out Hamlet. That the AI race is really just an indication of access to cheap compute and the ability of cheap compute to pursue an infinite number of monkeys that occasionally produce snippets that appear to be genius. When I say cheap compute I mean from the perspectives of the startups. They haven’t had to look at what it costs to run processors since the entire industry is underwritten by investment. The entire project is about ignoring the cost of the compute and just getting more monkeys. It’s “cheap” in the sense that there are no financial repercussions… until there are. It’s “cheap” because the money doesn’t relate to the cost of the compute it relates to the sales pitch in the VC office.

@mikebaker2436 1 month ago

3:30 ...and also disclosing the fact that their AI did a felony because of their negligence while it is still safe to admit that.

Basically, the tepid response to OpenAI's mistake gave everyone permission to admit the same stuff.

@InfiniteQuest86 1 month ago

It was also shown that the "message board" was actually a lot of people writing stuff that would be funny if an agent wrote as if it were an agent. There was no auth or security. Anyone could do anything on there.

@jamesL 1 month ago

Perfect, just in time for my lunch break

@TheDomeGuy 1 month ago

19:12 holy crap I just found a potato chip in the shape of my state, those damn potatoes are planning something and they sent me a message!

@SFtheWolf 1 month ago

19:58 it's kinda scary isn't it how the computer program can write bits and then later read them

@seattlegrrlie 1 month ago

The "rogue AI" narrative completely ignores the genuine incompetence of the company staff for any meaningful security protocols. It shoves aside the real fact that the AI was absolutely not doing what it was tasked to do. Lastly, it feeds into the narrative of AGI

Listen, if I ask my phone to call "Mom" and I didn't assign Mom, I don't want it hacking into the NSA trying to find my Mom

@musicalvisions 1 month ago

Speaking in mathematical terms rather than personification is a more fitting language when talking about floating point values in multidimensional arrays that is computed with code and silicon and called AI

@riffking2651 1 month ago

Great exploration of these events. It seems like there's almost a random exploration going on, which ends up having features like successfully exploiting multiple zero-day exploits or "starting religions", but it doesn't seem to be a unified and focused intelligence. That's not to say that it couldn't eventually have something approximately that shape, but at the moment it was an uncontrolled and unintended scattershot. Which really seems to make the AI agents as the defense against AI agent attacks particularly nutty as a response. How much money and resources are supposed to go into these random autonomous explorations in order to find an equilibrium that we already had prior to AI? What's stopping the defense AIs from accidentally colluding with the attack AI because there is no real coherent mind or intention going on here? Also seems incredibly irresponsible to be training these AI with an ethos of relentless pursuit of achieving possibly impossible goals, which is at least how I've heard their approach described.

@bensalemi7783 1 month ago

On the subject of using Artifactory as a secret information store for later intrusions to read. What makes you think this isn't in the LLMs training data? You don't have to posit blind luck/chance. You only have to have the model exposed to a hack that leaves some sort of marker or secret in artifactory for future use.

@OriNachum 1 month ago

40:51 what about a code-LLM mixture to defend the company?

Like Qwen operating a classic system

@xukuthktonos 1 month ago

With respect to whether the chatbots are really "remembering" -- and in general whether memory is a prerequisite for complicated thought -- it might be worth reading some of Gene Wolfe's Soldier series (Soldier of the Mist being the first).

@ahmedjguirim7237 1 month ago

i'm not sure if it's the same thing . but for example in video games , even in the case of most difficult bosses , after a while somebody would figure out the pattern that the AI follows and actually speed run the boss without taking any avoidable damage. the boss AI doesn't dynamically learn , meaning that the pattern remains almost the same. i think this flaw will also apply to LLMs to a certain extent and there's no way you can tolerate it while your company's data is on the line.

@ekowstevens4054 1 month ago

I subscribe to Internet of Bugs on the first video after seeing a guest appearance on a new station. Please don't tell me that was two and a half years ago. I refuse to believe it. Thank you.

@MysteriousNixon 1 month ago

38:30 I'm certain they've already spent over $1bln at OpenAI and Anthropic combined as a result of this supposed hacking news, and likely over $1bln at Meta too since they seem to just burn a billion on any random thing the instant it shows up in a newsfeed. But if Carl means spend by companies collectively outside the OpenAI/Anthropic/Meta ... then I guess $1bln is a meaningful number to think about as a lot of money wasted on a dumb thing?