NYC Council Hearing Exposes AI Safety Concerns and Regulatory Gaps
New York City Council held a rare hearing to scrutinize AI safety, corporate accountability, and proposed regulations, bringing together all 51 council members. Representatives from OpenAI, Anthropic, Google, and Meta testified alongside former employees who raised serious concerns about the development and evaluation of advanced AI systems.
Jacob Coxon, a former researcher at Anthropic and OpenAI, delivered a stark warning, arguing that the industry’s rapid-development culture is ill-suited for increasingly powerful AI systems. “On the current path, I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction,” he told lawmakers. Coxon criticized the startup mentality of moving fast and fixing problems later, calling it inappropriate for developing superintelligent AI. Daniel Kokotajlo, another former OpenAI researcher, echoed these concerns, warning that current evaluation methods may fail to detect dangerous AI behaviors before they escalate.
Representatives from OpenAI, Anthropic, Google, and Meta emphasized the benefits of AI and their commitment to safety, but lawmakers pressed them for more concrete answers about catastrophic risks. Morgan Dwyer of OpenAI admitted uncertainty about quantifying worst-case scenarios, a response that Council Speaker Julie Menin called “incredulous.” Google’s Alice Friend and Anthropic’s Logan Graham supported “smart regulation” but did not resolve disagreements over acceptable risk levels or how to measure them.
The Council is considering new AI regulations, including mandatory third-party validation for AI systems, human-controlled “kill switches,” and whistleblower protections. Proposals also address reporting AI safety incidents, emergency response planning, and the impact of algorithmic systems on city employment. Meanwhile, SpaceXAI, which received a subpoena but did not attend the hearing, faces separate legal pressure. UK Labour MP Jess Asato testified about her lawsuit against xAI, alleging privacy violations and harmful content generated by Grok.