State report looks at AI risks and rewards

Published On: November 27, 2023

California stands at the forefront of innovation, and in a landmark move, Governor Newsom’s Executive Order on Generative Artificial Intelligence (GenAI) has prompted an extensive 34-page report delving into the risks and rewards of integrating GenAI into the state’s governmental framework.

This report represents a pivotal step in leveraging the transformative power of GenAI while ensuring responsible and ethical utilization. California has the opportunity for GenAI to drive innovation, enhance service delivery, and uplift the lives of its residents.

One such example include’s GenAI’s ability to transform educational material into audiobooks, large print text, or braille.

“Accessible communications are a critical part of ensuring that government services can meet Californians where they are,” the report states. “The ability to meet the varying communication needs of persons with disabilities and reach Californians in their primary languages is a priority for improving government service delivery.”GenAI, distinct from traditional AI, holds unique capabilities that necessitate a novel approach to

implementation and evaluation. It promises substantial benefits, offering improved service delivery and increased accessibility to government programs. Yet, this revolutionary technology isn’t without its risks. Concerns include potential threats to democratic processes and biases to impacts on public health, safety, and the economy.

“[B]ad actors can leverage AI to engineer dangerous biological materials, AI chatbots could give consumers incorrect or dangerous medical advice, or GenAI systems used for drug discovery could create harmful substances. In sensitive domains like healthcare and public safety, GenAI requires careful governance to mitigate the risk of harm.”

One of the greatest concerns lingering within this field is the potential exacerbation of societal biases by GenAI. Its learning from input data has the capacity to amplify biases present in our society. Consideration must be given to prevent adverse impacts on various demographic groups, spanning regions, income levels, races, ethnicities, genders, ages, religions, and more.

Acknowledging the unprecedented nature of GenAI, collaboration becomes imperative. The report draws insights from frameworks like the National Institute of Standards and Technology (NIST) AI Risk Management Framework and international policies, emphasizing the need for a collective effort between states, federal entities, and international partners.

The report marks the inception of a multi-year, iterative process outlined in the Executive Order, mandating joint risk analysis, guidelines for procurement and training, impact assessments on vulnerable communities, workforce preparation, and regulatory impact evaluations.

The commitment to engagement echoes throughout the report. Academic, labor, community, and industry stakeholders are integral as the State embarks on piloting GenAI use cases. This collaborative approach aims not just for innovation but to establish robust guardrails safeguarding Californians and their data.

The state will regularly update the reports as significant developments are made in GenAI.

“Looking forward, the State will continue building off considerations laid out in the NIST framework for conventional AI as well as developments from the Biden Administration’s Executive Order on AI, and in ongoing partnership with the Legislature, community, academia, and technical experts. “


About the Author: Will Keys

Will Keys writes about technology issues for the GovReport. He is a graduate of the Reynolds School of Journalism at the University of Nevada, Reno. He can be reached at will at