The Future Society

ARE WE READY FOR AGI? This was the question we put — not hypothetically, but urgently — to 145 senior AI practitioners, diplomats and ministers at the 2nd edition of AI Safety Connect, held during #UNGA80 on September 25. Our event came just hours after the launch of the UN Global Dialogue on AI Governance (implementing Resolution A/RES/79/325), and days after the Global Call for AI Red Lines (https://red-lines.ai/) — now backed by over 300 prominent figures across science, diplomacy and civil society. And yet, despite all this momentum, one fact was impossible to ignore => Frontier AI companies are all openly racing toward Artificial General Intelligence (AGI), while global governance is nowhere near ready. Across sessions, we heard a converging set of messages repeated in different accents, from East and West alike: 4 Key Takeaways: 1/. AI Safety is no longer optional. Many warned that human-level AI systems could emerge within the decade. Few believe current trajectories are rooted in real scientific assurance or aligned incentives. Hope or complacency cannot b amount to a safety strategy. 2/. International cooperation is mandatory. Compute, data, talent and testing capacity are controlled by a tiny number of actors — pushing incentives toward speed and market capture, not prudence. We need shared rules, institutional guardrails and, above all, aligned incentives. And we need them fast. 3/. The world is now asking which AI Red Lines, not whether. Deception. Mass manipulation. Epistemic collapse. Uncontrollable systems. Catastrophic misuse or accidents. These are no longer abstract categories — they are pressing policy design questions. And they must be settled globally and proactively, not reactively and locally. 4/. We need an operational toolkit — not just declarations. Red lines mean nothing without verification, pre-deployment evaluation, incident reporting, and compute thresholds tied to capabilities. And not every issue needs the same diplomatic table: some coalitions will need to be broad, others truly restricted. A heartfelt THANK YOU to our stellar AI Safety Connect team starting with Cyrus Hodes for his vision & leadership; and to our stellar partners — The Future Society, FAR.AI, Mila - Institut québécois d'intelligence artificielle, UNDP, NYC AI Governance & Safety (NYAIGS), and the Permanent Missions of Singapore, Canada and Brazil — for helping place Advanced AI Safety firmly on the UN’s agenda. The awareness is now real. The question/challenge is whether we can turn momentum into machinery — before the next leap in capability. 👉 Follow AI Safety Connect (https://lnkd.in/eic8TMfV) #AISC #UNGA80 #AISafety #AGI #AIGovernance #GlobalCooperation Cc AISC core team Kay Kozaronek Uma Kalkar Leah Fayal 🙏🙏

ARE WE READY FOR AGI? This was the question we put — not hypothetically, but urgently — to 145 senior AI practitioners, diplomats and ministers at the 2nd edition of AI Safety Connect, held during #UNGA80 on September 25. Our event came just hours after the launch of the UN Global Dialogue on AI Governance (implementing Resolution A/RES/79/325), and days after the Global Call for AI Red Lines (https://red-lines.ai/) — now backed by over 300 prominent figures across science, diplomacy and civil society. And yet, despite all this momentum, one fact was impossible to ignore => Frontier AI companies are all openly racing toward Artificial General Intelligence (AGI), while global governance is nowhere near ready. Across sessions, we heard a converging set of messages repeated in different accents, from East and West alike: 4 Key Takeaways: 1/. AI Safety is no longer optional. Many warned that human-level AI systems could emerge within the decade. Few believe current trajectories are rooted in real scientific assurance or aligned incentives. Hope or complacency cannot b amount to a safety strategy. 2/. International cooperation is mandatory. Compute, data, talent and testing capacity are controlled by a tiny number of actors — pushing incentives toward speed and market capture, not prudence. We need shared rules, institutional guardrails and, above all, aligned incentives. And we need them fast. 3/. The world is now asking which AI Red Lines, not whether. Deception. Mass manipulation. Epistemic collapse. Uncontrollable systems. Catastrophic misuse or accidents. These are no longer abstract categories — they are pressing policy design questions. And they must be settled globally and proactively, not reactively and locally. 4/. We need an operational toolkit — not just declarations. Red lines mean nothing without verification, pre-deployment evaluation, incident reporting, and compute thresholds tied to capabilities. And not every issue needs the same diplomatic table: some coalitions will need to be broad, others truly restricted. A heartfelt THANK YOU to our stellar AI Safety Connect team starting with Cyrus Hodes for his vision & leadership; and to our stellar partners — The Future Society, FAR.AI, Mila - Institut québécois d'intelligence artificielle, UNDP, NYC AI Governance & Safety (NYAIGS), and the Permanent Missions of Singapore, Canada and Brazil — for helping place Advanced AI Safety firmly on the UN’s agenda. The awareness is now real. The question/challenge is whether we can turn momentum into machinery — before the next leap in capability. 👉 Follow AI Safety Connect (https://lnkd.in/eic8TMfV) #AISC #UNGA80 #AISafety #AGI #AIGovernance #GlobalCooperation Cc AISC core team Kay Kozaronek Uma Kalkar Leah Fayal 🙏🙏

The morning after I landed in NYC for the UN General Assembly High-Level Week, I was preparing for the press conference while watching the UNGA opening ceremony and listened to Nobel Peace Prize winner Maria Ressa, a signatory of the Global Call for AI Red Lines, announce our campaign in her speech. It was an incredible start to the momentum that the call would generate in the days and weeks that followed. Last week I had the opportunity to discuss the campaign’s goals with Philipp Sandmann in a 20-minute interview (linked in the comments). The Global Call for AI Red Lines has also been covered in 300+ media outlets, including The New York Times, BBC, NBC, El País, and Le Monde (links to a few in the comments). Hundreds of prominent figures have now signed, including 10 former heads of state and ministers, 15 Nobel Prize and Turing Award recipients, high-level scientists across the largest AI companies, and over 90 organizations. Engaging with experts at events from AI Safety Connect at the UN headquarters to side events across NYC has been extremely valuable. It's inspiring to see the interest and support for the call, and to gather insights on how to move forward. If you are working on topics related to AI red lines, I’d love to connect to discuss next steps, our strategy to turn red lines into reality, and opportunities for collaboration. A huge thank you to the amazing team at The Future Society in NYC and beyond. I’m grateful to have done this work alongside Niki Iliadis, Anoush Rima Tatevossian, Su Cizem, Mehdi Kocakahya, our broader TFS team, and partners from Centre pour la Sécurité de l'IA - CeSIA (with our great partnership with Charbel-Raphaël Segerie and Arthur Grimonpont) and Center for Human-Compatible AI at UC Berkeley (Stuart Russell and Mark Nitzberg), as well as the AI Safety Connect with Nicolas Miailhe, Cyrus Hodes, Leah Fayal, Kay Kozaronek, Uma Kalkar, Zeynep Güzey and more.