Gavin Newsom and Silicon Valley quash AI safety effort


California had a chance to set a national safety standard for artificial intelligence — but Gov. Gavin Newsom and Silicon Valley got in the way.

Artificial intelligence vaulted to the top of Sacramento’s agenda this year as legislative Democrats concluded Congress was too divided and dysfunctional to keep up with the fast-developing technology. Dominating the debate was a bill by state Sen. Scott Wiener, an ambitious San Francisco Democrat with an eye on Rep. Nancy Pelosi’s seat, that would require large AI models to be vetted for risks like launching a bioweapons attack.

But that appetite for action collided with vociferous opposition from some of the state's most formidable political players, including Silicon Valley heavyweights and House Democrats whose districts host AI headquarters.

They found a willing partner in Newsom, a tech-friendly former San Francisco mayor whose Sunday veto cemented his willingness to thwart the Democrat-dominated Legislature and reject bills regulating one of California’s core industries.

“He’s trying to push AI forward in California, and I think that bill would have put the brakes on it,” said Jim Wunderman, who leads a business coalition called Bay Area Council that includes major tech companies. “He’s tried really hard to be pro-tech as governor on a number of occasions. He gets that’s the major business of the state.”

The governor is also keenly attuned to California’s reliance on wealthy residents, with the tech industry’s boom-and-bust cycle fueling the state’s recent swings from an enormous budget windfall to a yawning deficit.

“Gavin was unlikely to sign a bill with that much organized local opposition, let alone from the industry that drives the most pronounced swings in personal income tax revenues in the state,” Kim-Mai Cutler, an investor who is active in San Francisco politics, said in a text message. Cutler noted the governor is “actually quite risk-averse.”

A Newsom spokesperson noted in a written statement that the Democratic governor signed numerous AI bills this month — including measures banning political deepfakes and giving performers more control of their digital likenesses — and said the governor “rejects the friend-or-enemy framing about the tech industry.”



”It’s not a binary choice,” said the spokesperson, Bob Salladay. “We can protect the public and foster innovation at the same time. California is very good at doing that.”

Wiener, a state senator with a penchant for tackling tough issues, labored for months to counter narratives that he was against artificial intelligence or that his bill would stifle innovation. Although he lamented Newsom’s rejection, he said he was proud to bring light to the issue.

"If SB 1047, by failing, ends up spurring a productive conversation and actual, tangible policy that promotes innovation and protects the public, then that's a win in the long run,” he said in an interview.

California's debate is far from over. Newsom is promising to craft follow-up legislation, and Wiener has suggested he’ll pursue the issue next year. But the veto raises larger questions about the national future of an idea that was front-and-center for Washington just a year ago.

“[The bill] would have spurred Congress to act," said Jason Green-Lowe, executive director at the Center for AI Policy, a nonprofit with ties to the effective altruism movement. "Now there's even more need for Congress and the next president to take a stand on AI safety. But this California bill could have bought us some time and showed the way forward."

The rapid advent of AI has divided the tech world between people who want to move as fast as possible and skeptics who fear the technology could wreak havoc. Wiener cultivated support from researchers who pioneered the field, worked on amendments with the AI firm Anthropic and found an unlikely ally in Elon Musk, who backed the bill despite calling Wiener an “utter scumbag.” Hollywood celebrities urged Newsom to sign it.

But an ever-growing collection of Silicon Valley’s most influential players fought the bill as it moved in Sacramento. That included leading artificial intelligence companies like Meta and OpenAI and wealthy venture capitalists like Marc Andreessen, Garry Tan, and Ron Conway — a fixture in San Francisco politics who has supported both Newsom and Wiener. Foes signed up blue-chip lobbying firms, some of whose principals have ties to Newsom.

“This did start as an intra-tech fight, and by the end it was clear that that’s not all this was,” said Nathan Calvin, senior policy counsel for the Center for AI Safety, which sponsored the bill. But ultimately, he said, supporters were "up against an industry that’s not used to substantive regulations, and there’s quite a lot of energy around keeping that in place.”

Criticism from tech workers exploded on social media. Startup founders grilled Wiener at a July event hosted by Y Combinator, the startup incubator that was formerly led by OpenAI CEO Sam Altman. Google Brain co-founder Andrew Ng made a surprise appearance to castigate the bill.

But those concerns couldn’t stop the bill’s steady advance through the Legislature. With each vote, critics increasingly put their hopes in Newsom. The veto was one of Newsom’s least surprising moves of the year.

The governor maintains deep ties to a tech sector that he sees as one of California’s economic pillars and a vital source of tax dollars. He regularly stresses the need for California to maintain its competitive edge on artificial intelligence and warns against over-regulation. He has made a similar argument in rebuffing tech-constraining bills like a clampdown on self-driving trucks.

“We dominate this space, and I don’t want to lose that competitiveness,” Newsom said earlier this month in an interview with Salesforce CEO Marc Benioff, a close personal friend. “The impact of signing the wrong bills over the course of a few years could have a profound impact on our competitive strength.”

As an olive branch, Newsom paired his veto with a promise to convene a working group to develop “common sense guardrails” but didn’t specify what those would look like or when they would go into effect. The working group, he said, will be led by leading AI academics including the “godmother” of AI, Stanford professor Fei-Fei Li, who interviewed Newsom on stage earlier this year and went on to openly oppose the bill.



That seemed to only further irritate the bill supporters, who point out that Li had an AI startup of her own.

“It’s ironic to put people in charge of trying to effectuate some kind of common-sense safety guardrails who have been very outspoken against it,” said Teri Olle, director of Economic Security California, which backed Wiener’s bill.

“It makes me skeptical that that’s going to result in anything meaningful,” Olle said.

Li, in a statement, said it's crucial for the effort to include a range of people who are working in the emerging sector.

"It would be ironic to come up with common-sense safeguards without involving the experts who are building in AI, which includes leaders from academia and the start-up ecosystem," she said. "So if it is a conflict of interest for leaders of start-ups and academia in AI to have a seat at the table in discussions of tech policy, then the same is true of the leaders of Big Tech who have continued to have the biggest seat at the table."

Another blow to Wiener and his allies came from Capitol Hill. Bay Area Democrats rallied against the bill, starting with Rep. Zoe Lofgren — the top Democrat on the House’s main science committee — and culminating with Pelosi, who is close to Newsom. Her intervention was widely seen as an effort to undercut Wiener as he rumbles toward a succession battle with the ex-speaker's daughter.

Lofgren wrote to Newsom and to state Assembly Speaker Robert Rivas, whose district overlaps with hers, and personally reached out to other lawmakers ahead of a key vote urging them to reject the bill. Less than an hour after Newsom vetoed the bill, Pelosi praised him in an X post noting “AI springs from California.”

Tech luminaries showered Newsom with praise soon in the wake of the veto. Democratic megadonor Reid Hoffman lauded the governor’s “leadership.” Conway pledged to work with Newsom on crafting a “a responsible and safe AI regulatory framework.” Tan, who has become a combative and key figure in San Francisco’s internecine Democratic struggles, said he was grateful to the governor.

Catie Stewart, a San Francisco-based political consultant who formerly worked for Wiener, said most elected officials struggle to overcome the magnitude of opposition the bill unleashed — particularly those, like Newsom, who want to keep the road to the White House unobstructed.

“When you’re going up against people like a Ron Conway or other mega-billionaires, it’s just hard,” Stewart said. “If you’re looking at a presidential run in the next four to eight years and you’ve pissed off a lot of tech people, a lot of tech donors particularly, that’s a scary thing.”

Mohar Chatterjee contributed to this report.



Comments