Although safety concerns about the rapid growth in capacity of artificial intelligence alongside growing difficulty in controlling the behavior of the most advance agents was on the agenda for the recently concluded summit between Presidents Xi Jinping of China and Donald Trump of the United States--the two men in the world today most able to regulate this technology or to block regulation--it appears no meaningful progress was made at their meeting. Meanwhile, the pace of AI development, the looming possibility of recursive self improvement when AI becomes fully capable of making new and more powerful artificial intelligence, and the increasingly alarming signs of misalignment between what the human in the labs making AI instruct their agents to do and what the AI agents actually do all seem to point to the idea idea that we are at a critical moment for AI regulation, one in which further delay may allow the technology to develop to the point where it will be difficult or impossible for us to control in the future. Why would this be deeply dangerous to humanity? And why are we seemingly unable to respond appropriately to the threat? These are the urgent questions we discuss in today's class.
Although safety concerns about the rapid growth in capacity of artificial intelligence alongside growing difficulty in controlling the behavior of the most advance agents was on the agenda for the recently concluded summit between Presidents Xi Jinping of China and Donald Trump of the United States--the two men in the world today most able to regulate this technology or to block regulation--it appears no meaningful progress was made at their meeting. Meanwhile, the pace of AI development, the looming possibility of recursive self improvement when AI becomes fully capable of making new and more powerful artificial intelligence, and the increasingly alarming signs of misalignment between what the human in the labs making AI instruct their agents to do and what the AI agents actually do all seem to point to the idea idea that we are at a critical moment for AI regulation, one in which further delay may allow the technology to develop to the point where it will be difficult or impossible for us to control in the future. Why would this be deeply dangerous to humanity? And why are we seemingly unable to respond appropriately to the threat? These are the urgent questions we discuss in today's class.