AI Governance Needs Radical Optionality

How should governments regulate the most advanced AI systems? One possible answer is that they should not. Libertarian-minded writers have made the case for a culture of “permissionless innovation” for AI development, in which the role of government would be limited to enforcing existing laws and facilitating industry self-regulation with “soft law” tools such as voluntary standard-setting. Another possibility, more in vogue across the aisle and the pond, would invoke the precautionary principle, using heavy-handed regulation to restrict the development of frontier AI models until developers can adequately prove that their systems are safe.

I think there’s a better approach. In a new essay, my co-author Christoph Winter and I make the case for a governance strategy that we call “radical optionality.” The idea is simple: governments should avoid over-regulation in the short term while building up the institutional capacity needed to competently regulate extremely advanced or “transformative” future AI systems when and if these systems come into existence. The point of this approach is to maximize optionality by providing our institutions with tools that can be used to respond to a wide range of foreseen or unforeseen future developments.

In this piece, we explain the rationale behind maximizing optionality while AI’s future impacts remain uncertain. We also outline some example measures that governments can take and explain how this approach dovetails with other proposals for AI governance.

Addressing AI Risks Under Uncertainty

Leading AI researchers in academia and industry have claimed that advances in AI capabilities may soon produce “AGI,” “artificial superintelligence,” “powerful AI,” or some similar term. If you are certain that these statements are hype, and that such advanced AI systems will not arrive during our lifetimes, I won’t try to convince you otherwise; enough ink has been spilled on the subject that I’m not optimistic about my ability to contribute anything new. But if you think there is even a small chance that these predictions materialize, or if you find them at all credible, we think that the argument for radical optionality is overwhelmingly strong. The argument goes as follows.

/inline-pitch-cta

AI’s future impacts are highly uncertain. Assume that there is some possibility of transformative AI systems being invented within the next, say, 15 years or so. Most of us are extremely uncertain about exactly how and when this will happen, what the characteristics and tendencies of these systems will be, what benefits they will offer society, and what risks to public safety and national security they will create. Under some assumptions, these systems will be mostly harmless and highly beneficial, because the companies creating them will have incentives to make them safe and broadly aligned with human preferences. Under other assumptions, these systems will be dangerous and difficult to control—perhaps even capable of causing human extinction if the right guardrails are not put in place. Maybe securitization is inevitable and the U.S. government will soon spring into action and develop these systems behind closed doors as part of a clandestine military project. Alternatively, perhaps development will happen in a decentralized and democratic way.

The question of regulation seems to present a tradeoff between innovation and security. Debating which of the scenarios described is most realistic can be valuable, but ultimately only ideologues claim to be certain about the future course of a poorly understood emerging technology. The rest of us have to make important decisions about what to do while acknowledging substantial uncertainty. On the one hand, restrictively regulating AI companies would slow down innovation, potentially depriving society of some of the benefits of technological progress. On the other hand, it is possible that well-designed regulations could mitigate the real risks that AI systems pose, both now and in the future. How should we think about this tradeoff between innovation and security?

Measures that maximize optionality can improve security without hindering innovation. We think that this framing misses an essential point: there are steps governments can take now that would increase security without any significant cost to innovation. At the top of the list are light-touch information-gathering authorities like whistleblower protections, reporting requirements, and transparency mandates. Government agencies thrive on a diet of information; it has been said that “information is the lifeblood of good governance.” Authorities that increase the government’s access to important information about AI risks—and allow the relevant agencies to develop expertise in securely processing and interpreting such information—are foundational building blocks for future governance efforts. Mechanisms for securely and intelligently sharing information within government, and (when appropriate) between governments, are similarly foundational.

/odw-inline-subscribe-cta

Building capacity directly is also important. First and foremost, this means enabling the relevant regulatory bodies to hire and retain elite talent. Meta’s recent hiring spree, featuring 10-figure compensation package offers for top AI talent, is an example of what it looks like when an organization takes the prospect of transformative AI seriously. Governments will likely be unable to compete with the salaries on offer in the private sector, but reforms to processes for government hiring and contracting of AI talent are nevertheless needed in both the U.S. and the EU. The early successes of the UK’s AI Security Institute, which receives 10 times the funding of its U.S. counterpart despite the UK’s relatively modest GDP and industry relevance, demonstrates the importance of cultivating talent in government.

The full-length essay discusses a number of other optionality-increasing policy decisions, such as incentivizing lab security, avoiding premature and overbroad preemption of state laws, and building out an ecosystem for model assessments and evaluations. But the important thing is to recognize that security and innovation are not conflicting priorities, because there are ways to increase optionality without creating any significant barriers to technological progress.

Supporting Both Security and Innovation

Radical optionality is compatible with other proposals for AI governance. Radical optionality is by no means the first AI governance framework to recognize that governments have an important role to play while also acknowledging that overly restrictive regulation could hinder innovation. Dean Ball and Gillian Hadfield and Jack Clark have proposed sophisticated private governance regimes in which the government would certify an ecosystem of private regulators competing to offer efficient and nimble regulatory services to companies on an opt-in basis. Gabriel Weil has argued that a well-designed tort liability regime, featuring insurance requirements, punitive damages, and strict liability for certain harms, could force AI companies to internalize any risks generated by their products. And Cary Coglianese has advocated for a system of management-based regulation, requiring AI companies to take risk mitigation measures but allowing them broad discretion over what measures to implement and how. I view these proposals as consistent and compatible with radical optionality; tort liability, management-based regulation, and private governance mechanisms are valuable tools for maintaining and increasing optionality.

Under most worldviews, radical optionality is preferable to the status quo. Of course, not everyone will agree that an optionality-maximizing approach is wise or sufficient. If you confidently believe that the safety benefits of a restrictive AI regulatory regime would outweigh the costs to society of slowing innovation, it is reasonable to suggest that anticipatory regulation is needed. From this perspective, radical optionality does not go far enough, but it would still be preferable to the status quo. On the other hand, from a libertarian perspective, building up government capacity to regulate and promising that it will not be used prematurely might look a lot like giving the government a hammer and promising that agencies will not start hallucinating nails. I can’t promise that there is no chance of new authorities being abused, or that everyone will agree on when dual-use AI systems have become so advanced that regulating them is a national security imperative. But I do expect building government capacity to benefit AI companies as well as the public in the long term. If rapid progress in AI capabilities research gives rise to a surge in public demand for regulation at some point in the future, as some writers have predicted, companies might prefer for the government to have the option of regulating in a competent, targeted manner.

At my organization, the Institute for Law & AI, we spend a lot of time thinking about how advanced AI systems should be governed in the present and in the future. Radical optionality is a sort of organizing principle and guiding philosophy for that research and consulting work. When deciding what projects to work on, what bills to offer feedback on, and what policies to push for, the question of what approach will maximize optionality is typically one of the foremost considerations. In publishing this paper, I hope to convince at least a few people to adopt this framing, to recognize the importance of optionality, and to begin viewing security and innovation as compatible rather than conflicting priorities.

‍

See things differently? AI Frontiers welcomes expert insights, thoughtful critiques, and fresh perspectives. Send us your pitch.