Published Work

Governor Vetoes California's AI Safety Bill: What's Next?

|
Kevin M. Benedicto
| Legaltech news

On Sept. 29, 2024, California Gov. Gavin Newsom vetoed Senate Bill 1047, a comprehensive legislative proposal to regulate the rapidly growing AI industry. The proposed law included safeguards and policies to prevent "catastrophic" harm from increasingly powerful AI models.

In vetoing the bill, Newsom highlighted the need to balance the regulation of AI technology with the risk of stifling innovation. While SB 1047 did not become law, the battle to regulate AI is only beginning, and the proposals in SB 1047 will reverberate in California and elsewhere as lawmakers consider what to do next.

Background of SB 1047

Drafted by state Sen. Scott Weiner of San Francisco, SB 1047 was one of the most significant AI safety bills to pass any legislature in the U.S. It focused on developers of expensive, large-scale AI models that pushed the limits of technology, terming these "frontier" models. The bill would have imposed legal liability for developers of frontier models for any "catastrophic" harms those models cause. As a result of the bill, larger AI companies would have been required to put certain safeguards and policies in place to prevent future "catastrophic" harms. The bill also would have created new government entities, including a Board of Frontier Models, to review the activities and certifications of covered AI companies, including assessing and auditing compliance with the law. It also would have created whistleblower protections for tech workers at AI companies who wanted to provide information to the authorities.

While the bill passed with wide bipartisan majorities in both houses of the California legislature, it prompted immense public debate in the wake of its passage. Much of the technology industry opposed the passage of the bill, arguing that its restrictions were too onerous and that it would stifle innovation. By contrast, several notable academics and researchers specializing in AI safety publicly supported the proposal.

When he issued his veto, Newsom acknowledged the need to regulate the industry but disagreed with the specific provisions and proposals in SB 1047. The governor also cited the fact that "California is home to 32 of the world's 50 leading AI companies" as both a reason why California should lead on AI regulation and a reason to balance regulation with the goal of fostering innovation.

What Is Next in California?
While SB 1047 did not become law, the governor signed several other bills that regulate the AI industry, and it is apparent that California will continue to assert its influence and leadership on AI regulation. Several key proposals became law in 2024.

Regarding election protection, Newsom signed into law several bills (AB 2655, AB 2839, and AB 2355) that prohibit deceptive content in election materials and require disclaimers for AI-generated content in certain political advertisements. These proposals seek to "combat the harmful use of deepfakes in political ads and other content, one of several areas in which the state is being proactive to foster transparent and trustworthy AI."

The governor also signed AB 2602 and AB 1836 into law, which enact new restrictions on the use of digital replicas of performers. The issue of digital replicas of performers was a central concern in the recent Hollywood strikes. AB 2602 makes certain contracts for the creation and use of digital replicas unenforceable unless certain safeguards are met, and AB 1836 prohibits the use of digital replicas of deceased individuals without the consent of the family or estate.

Additionally, recent amendments to California's comprehensive privacy laws, the California Consumer Privacy Act (CCPA) and the California Privacy Rights Act (CPRA), were also passed and signed into law. These amendments clarify that the CCPA and CPRA's protections of personal information apply to AI technologies and models. They also provide protection for potential future use of consumers' neural data, such as brain and nervous system functions, in addition to existing protections on biometric data. 

Proponents of SB 1047 have vowed to advance similar proposals in future legislative sessions.

Impacts Beyond California
Jurisdictions nationwide and overseas are similarly grappling with how to regulate the fast-growing AI industry and striving to balance the same considerations of encouraging innovation while limiting risk. The provisions contained in SB 1047 will likely make their way to other jurisdictions and be considered by policymakers, and the impact of the proposal and its veto may be far reaching. And while SB 1047 did not become law, other jurisdictions are not waiting to pass their own AI regulations.

Colorado became the first U.S. state to adopt wide-reaching AI legislation, which imposes new prohibitions and disclosure requirements on AI-powered algorithmic decision-making and imposes new duties and responsibilities on developers and deployers of AI-powered decision-making tools. Three states (Illinois, Massachusetts and Ohio) have legislation regulating AI currently in committee in their legislatures, and 13 states introduced or considered some form of AI regulation this year.

Policymakers will certainly look to the details (and ultimate veto) of SB 1047 as they craft proposals and policies to regulate AI going forward. The federal government is also becoming increasingly involved, with the White House issuing the first-ever executive order on AI, and several federal agencies issuing regulatory rules and bringing litigation against AI companies.

Internationally, the most high-profile AI regulation is the recently passed EU AI Act, a sweeping and comprehensive legal framework for the regulation of AI in the EU. After the EU adopted its comprehensive privacy framework in 2016 with the General Data Protection Regulation, many jurisdictions around the world followed; it remains to be seen if the EU AI Act could have similar implications.

Conclusion

The unprecedented explosion in AI technology and the AI industry has left regulators and lawmakers racing to regulate them. While California did not adopt the regulations of SB 1047, the state has continued to push AI regulations at a breakneck pace and is unlikely to surrender its influence and leadership position in the industry. The race to craft comprehensive regulations on AI technology is on and will only accelerate from here.

Kevin M. Benedicto is counsel with Redgrave in San Francisco. He develops discovery strategies in complex litigation matters and works with clients on issues surrounding cybersecurity, data privacy, cyberinsurance, compliance with new laws and regulations, and data incident response. Kevin has also represented and advised clients in the cloud computing, gaming and esports, and financial technology (fintech) sectors. 

Related people

Explore Our Team