Photo: Bettmann / Getty Images
This past September, California Gov. Gavin Newsom signed SB 53, the Transparency in Frontier Artificial Intelligence Act, a first-in-the-nation law whose aim is to promote artificial intelligence innovation while protecting public safety by ensuring AI models are safe and secure, trustworthy and transparent.
The law is meant to put in place some "commonsense guardrails" on the development of leading-edge algorithms, taking a "trust but verify" approach to model development.
Among other provisions, the law requires large developers of advanced "frontier" AI models to publicly publish frameworks describing how they have incorporated national and international standards, and industry-consensus best practices, into their frameworks.
It calls for the creation of a new mechanism for frontier AI companies and the public to report potential critical safety incidents to California's Office of Emergency Services and protects whistleblowers who disclose significant health and safety risks posed by such models.
The legislation calls on the California Department of Technology to recommend appropriate updates each year based on emerging standards, evolving stakeholder input and technological developments.
And it sets up a new consortium, CalCompute, within the states to develop a framework for ensuring development and deployment of AI models that are "safe, ethical, equitable and sustainable."
"California has proven that we can establish regulations to protect our communities while also ensuring that the growing AI industry continues to thrive," said Newsom in a Sept. 29 statement.
"This legislation strikes that balance. AI is the new frontier in innovation, and California is not only here for it – but stands strong as a national leader by enacting the first-in-the-nation frontier AI safety legislation that builds public trust as this emerging technology rapidly evolves."
With the new law, the Golden State has now taken on the role of "lead U.S. regulator of AI," as law firm Latham & Watkins LLP noted in October.
"SB 53 represents a paradigm shift in the regulatory treatment of advanced AI development," attorneys said, noting that AI companies "will need to undertake significant compliance preparation" – mapping model assets, creating and implementing risk management frameworks, developing processes for incident monitoring and regulatory reporting, and more.
"Penalties are steep – just a single violation is subject to a seven-figure fine – so it is critical that frontier developers that fall within SB 53’s scope work closely with counsel to develop a fulsome compliance program."
Balancing innovation and regulation
Early response from the healthcare industry has been positive about what the new law means for setting up some AI guardrails.
"This new law out of California is going to make a difference not only within the state but also globally, given the way in which frontier model LLMs are relied upon by thousands of other companies in their own AI efforts," said Leigh Burchell, VP of policy and public affairs at Altera Digital Health, in a statement shared with Healthcare IT News.
"The transparency that will now be required will be helpful to software developers – even those not directly implicated by the law – that want to ensure the AI tools they roll out include the safeguards necessary to deploy technologies that are both innovative and safe," she added.
"The law strikes a balance between supporting innovation and implementing guardrails for the responsible development of AI systems," said Shini Pattni, head of legal at Lindus Health, which develops tools and strategies for clinical trials.
"Many of the largest AI companies already document their safety protocols, so SB 53 should create consistency in the industry rather than creating an onerous compliance burden. It is particularly encouraging to see some concrete steps towards encouraging innovation."
Pattni applauded the fact that, with CalCompute, the law establishes a "public cloud computing cluster, which would support public AI research and democratize access to advanced computing resources. This is welcome news for startups, which may otherwise be unable to compete with the massive infrastructure of larger companies."
California isn't done with its AI regulation either, of course. For example, just this past month, Newsom signed SB 243, billed as a first-of-its-kind law focused on AI "companion chatbots" that offer natural language processing and adaptive humanlike social interaction.
"The law is designed to protect users – especially minors – from being misled about the nature of their interactions and from exposure to potentially harmful or inappropriate content," according to a brief from Skadden, Arps, Slate, Meagher & Flom LLP.
It contains an array of disclosure requirements, safety protocols and reporting rules, and could mean "civil claims and potentially steep damages" for AI developers who don't comply.
As recently as this past summer, it was looking as if the "One Big Beautiful Bill Act," which was signed into law by President Donald Trump on July 4, would contain a provision banning states from regulating AI for the next decade.
That language was eventually removed from the legislation after states pushed back.
'There's a lot to figure out'
States, after all, have been the key driver so far in setting some guardrails around fast-evolving AI technologies, in healthcare and elsewhere.
As Sarah Jaromin, policy specialist at the National Conference of State Legislators, a bipartisan membership organization that represents all 50 states, D.C. and the territories, explained to us recently, "The states are really involved" in crafting AI-focused legislation focused on an array of challenges and use cases for the technology.
The NCSL is tracking more than 1,000 bills related to AI broadly across the states – and more than 200 related specifically to the use of AI in healthcare, she said. About two dozen of them have been signed into law.
Jaromin spoke on a panel in September at the HIMSS AI Leadership Strategy Summit in Chicago. Along with policy experts and a state legislator from Tennessee, the panel offered an overview of the fast-changing landscape of AI-related lawmaking.
If states are the "laboratories of democracy," they're also fast becoming key proving grounds for many new AI regulations as the U.S. Congress concerns itself with other matters.
"There's an awful lot happening out there, and a lot of movement and a lot of amendments to each of the bills as they're moving through the legislative process," said Tina Joros, VP of policy and innovation at Veradigm.
If there are a thousand AI bills that have been introduced at the state level, as recently as last year that number was between 500 to 700, depending on how you counted, said Joros, who also serves as chair of the AI Task Force of the Electronic Health Record Association.
"I mean, it's just a flood. We've seen it too at the federal level, there's some activity there. But I think states are responding by saying, 'We're going to do our best to take care of the people that live in our state.' I think that's a good thing. But there's a lot to figure out. So there's lots of bills out there."
Republican State Senator Bo Watson of the Tennessee Legislature said state lawmakers welcome any assistance technology experts can offer about fast-evolving AI innovation.
"For those of us on the policy-making side, we need your help to determine how much, how little we need to be enforcing legislation and enforcing regulation on an emerging sector in the technology space," he said.
Watson asked healthcare and IT leaders to "lean in and help your legislators back in the state that you came from to help make us make really good decisions about moving forward, because it's evident by the number of pieces of legislation you're being introduced: A lot of states are grabbing different ideas and different concepts."
And no question: "One thing that states do that's really good is that we learn from each other," he added. "We share a lot of information between each other, and then we share a lot of legislation between the states."
But overall, the technology is "developing at a faster pace than the private sector can keep up with, and certainly if the private sector can't keep up with it, the public sector is not going to be able to," said Watson. "We're trying to figure it out."
Policy priorities
"People get surprised when I tell them that most legislators in the states are part-time," said Jaromin. "They have full-time jobs outside of the legislature. While Senator Watson is an expert on this, there are a lot of our members who aren't necessarily experts in AI or experts in health.
"So my plea to all of you would be to engage with NCSL or your members in your states to think about what you can educate them on and help them to make informed decisions."
Watson said one of the biggest challenges he and his colleagues need to grapple with is the energy demand of AI data centers.
Other priorities are focused around health plans and prior authorization.
"One-third of the legislation that I track this year is related to either prohibiting or monitoring the use of AI within utilization management processes by payers," said Jaromin – citing recent bills that either prohibit the use of AI in decisions around medical necessity or requiring humans to be the final arbiters.
"Another new one that we're seeing is the regulation of chatbots," she added. "There's some in Utah, there's some legislation in New York, just looking at whether they have to disclose that it isn't a human."
"There's a bill in front of the California governor right now that says it is not a defense that AI did it," said Joros, in apparent reference to SB 243. "So if you have a chatbot that is consumer-facing, if this law comes in, it will not be a defense for you to say, 'Hey, we put the tool out there. The person got the information. Sorry, we didn't know what it would say.' You're going to have to take responsibility for that.
"That liability question is a big one," she added. "Who is responsible if this chatbot does use me and gives you bad information? What if it has a serious consequential impact on your life or your health or some other aspect of your life? Who's responsible?"
'Infrastructure of our political system'
It shouldn't be a surprise that the states are taking the lead on AI regulation, said Watson.
"States do almost all the licensure for healthcare professionals," he said. "If AI is being used in the diagnostic space, who owns that diagnostic space? Do we have to, at some point in the future, come back and license AI in certain ways because it's using certain information to provide other information and so forth.
"Licensure is a heavily combated area when you're in the legislature," he added. "Everybody wants to protect their turf, but yet AI will work across multiple disciplines, and then ultimately what license has responsibility for the decisions that you're going to make from AI."
Put simply, "state government is the infrastructure of our political system," said Watson. "The federal government is very entertaining. It's a 24-hour show. But state and local governments have a lot of responsibility.
"And so it is critically important that you engage yourself in your state government and you engage yourself in your state legislation," he added. "Because ultimately, they will be making 95% to 98% of the decisions that will have an impact on your business model."
The HIMSS AI & Cybersecurity Virtual Forum is free to attend on Nov. 18. Learn more and register.
Mike Miliard is executive editor of Healthcare IT News
Email the writer: mmiliard@himss.org
Healthcare IT News is a HIMSS publication.
