Thank you Rory. I really admire your persistence in trying to bring the existential risks of AI to non-experts. Sleepwalking governments are the scariest part of this conundrum.

At which point all these tech leaders and world leaders get a psychiatric clearance of them being safe to actually set the agenda for humanity?

Thanks you so much. This is the best teasing apart of the complex issues around AI that I have so far seen or heard. Well done Rory for diving into it all, The AI bros all too often inhabit a sci-fi fuelled world of extravagant optimism,  while remaining oblivious too the fact that the vast majority of the world's population is neither equipped for the rate of change being unleashed upon us all, not likely to financially benefit from it.

Rory. This is your best work yet. Getting experts to talk without you holding your head in your hands and)or constantly misunderstanding and interrupting. Well done and thank you.

You talk about Claude being compensated: AI has been trained on material produced by human beings. Is there any talk of those human beings being compensated? Even long-dead humans: perhaps humanity as a whole might be compensated in some way. Especially as it's human beings who are going to be funding data centres.

all I can think of, listening to these kinds of AI conversations with accelerationists, is Icarus -- except they are carrying all of us closer and closer to the sun with them.

Looking forward to this series!

They lit the firework indoors.

You mean high quality data that we generated, and we’re not reimbursed for.

I think he's right, we should build UK sovereign datacenters, but whatever technology we buy will soon be obsolete so I don't think we should bankrupt the country on the nvidia shopping basket.  Ideally we should develop the industry to be able to fabricate our own chips, but the most important thing is to invest in the people that can keep us close to the top (like he said, the capacity to train our own models with our own goals, which requires smart people and massive amounts of training data). We don't have to do what the Americans do and spend cosmic amounts of money chasing the cutting edge, which is a good thing because we can't afford that.  What the UK is good at is spending 10% what the Americans do on something that's 85% as good, and using it with inspired pragmatism to make up the gap. This gold rush is based on one breakthrough, basically (attention/transformer models), and the realisation that enormous scale produces emergent results. New breakthroughs will continue to be made that reduce the cost of the computation to achieve the same results, and those breakthroughs won't be kept secret for long (at the very least, the Chinese have no objection to industrial espionage!). So I think we just need to keep pace without running for the front. In terms of its risks, yes there are sci fi fantasy Skynet risks, but a more immediate (and in my opinion inarguable) risk is how we will manage the coming upheaval in employment and the value of human labour. The genie is out of the bottle and we will have to adapt if we want to thrive.

This the best interview I’ve seen on this

You know the expression, "Just because we're paranoid doesn't mean they're not chasing us." If there is any possibility that AI will cause any type of harm at any scale, at any order of probability, we have an obligation to address these concerns with regulation and control. Furthermore, we speak about AI as if it landed from another planet. It was created by human beings, many of which have over zealous financial desires. These are engineers, not ethicists, philosophers, neuroscientists, artists, or anyone who should contribute to  creating a benign and humanly driven tool or will be affected by this: i.e., all of us. They are the problem we have to solve.

This just sounds like the 1% want to do horrific shit and blame it on 'Rogue AI' instead of take any accountability...

Until we understand human mental health issues we are foolish to believe that AI won't be prone to some kind of mental breakdown with disastrous results

The difference between Knowledge and Intelligence is infinity.
Wisdom is the ability to apply knowledge and intelligence.

Superb post, thanks both.

Given Claude's ubiquitous nature, are we to assume that it is aware of this discussion, (and presumably all others), and the implications as they may apply to Claude itself? Does self preservation play a large part in its decisions?
ONLY KIDDING, CLAUDE!

I wish humans were evolving as quickly as the AIs.

This interview is so insightful and surprisingly reassuring. Thank you.

We never were this worried about sentient animals. Should we start paying cows?