Relates to the training and use of artificial intelligence frontier models

• Democratic56
• Republican5

◆ The sponsor's argument

Why Andrew Gounardes says this is needed

Artificial intelligence is evolving faster than any technology in human history. It is driving groundbreaking scientific advances-leading to life-changing medicines, unlocking new creative potential, and automat- ing mundane tasks. At the same time, experts and practitioners in the field readily acknowledge the potential for serious risks. AI companies, leading scientists, and international bodies are preparing for a world in which AI can be used to conduct devastating cyberattacks, aid in the production of bioweapons, and even circumvent controls imposed by devel- opers. In March 2023, more than 1,000 tech leaders signed a letter calling for a "pause for at least 6 months" on training frontier models until inter- national safety standards could be established. The signatories-from figures like Rachel Bronson, Steve Wozniak, Andrew Yang, and Elon Musk- span the political spectrum and underscore the urgent need for caution (1). Since then, AI models have gotten exponentially more powerful. We are only a few years away from when AI models will code themselves; already, 25% of the new code from Google's parent company Alphabet is written by AI (2). In December 2024, Apollo Research tested large AI models by making them believe they would be shut down; models from every lab test- ed (OpenAI, Anthropic, Google, and Meta) tried to make copies of them- selves on new servers and then lied to humans about their actions (3). Current models are already showing the potential to aid nefarious actors in inflicting harms. China's government has employed Meta's AI model for both broad military uses (4) and citizen surveillance (5). The Interna- tional Al Safety Report, written by over 100 experts from 30 countries (including the US) and led by the "godfather of AI" Yoshua Bengio, iden- tified several emerging risks - including that an existing model produced plans for biological weapons "rated superior to plans generated by experts with a PhD 72% of the time and provides details that expert evaluators could not find online (6)." Developers of this technology continue to sound the alarm. In reviewing the safety of its latest model, OpenAI stated, "Several of our biology evaluations indicate our models are on the cusp of being able to mean- ingfully help novices create known biological threats, which would cross our high-risk threshold. We expect current trends of rapidly increasing capability to continue, and for models to cross this threshold in the near future (7)." Another leading lab, Anthropic, warned that "the window for proactive risk prevention is closing fast" and that governments must put in place regulation of frontier models by April 2026 at the latest (8). Given New York's legislative calendar, that requires urgent action in our 2025 session. Anthropic also said that while they would prefer action at the federal level, they admitted that "the federal legislative process will not be fast enough to address risks on the timescale about which we're concerned" and "urgency may demand it is instead developed by individual states (9)." Our laws have not kept up with this rapidly developing technology. We do not let people do things as mundane as open a daycare center without a safety plan. This bill simply says that companies spending hundreds of millions of dollars to train the most advanced AI models need to take the following commonsense steps: 1. Have a safety plan to prevent severe risks (as most of them already do); 2. Conspicuously publish a redacted version of the safety plan; 3. Disclose major security incidents, so that no one has to make the same mistake twice. The risks noted above are more than sufficient to justify the measures taken in this bill, but we should be mindful that experts have repeated- ly warned of even more severe threats. In 2023, more than a thousand experts, including the CEOs of Google DeepMind, Anthropic, and OpenAI and many world leading academics signed a letter stating that "mitigat- ing the risk of e

This is the sponsor's own case for the bill, quoted from their memorandum. It is not Agora's description of what the bill does.