Risk, Uncertainty and AI: Non-probabilistic methods for anticipating and preventing AI risks

Gutfraind, Alexander; Bier, Vicki M.

Permalink

https://hdl.handle.net/2142/121857 Copy

Description

Title

Risk, Uncertainty and AI: Non-probabilistic methods for anticipating and preventing AI risks

Author(s)

• Gutfraind, Alexander
• Bier, Vicki M.

Issue Date

2023

Keyword(s)

• Risk analysis
• Artificial intelligence (AI)
• Machine learning
• Non-probabilistic methods

Abstract

The rapid advancement of artificial intelligence (AI) has created a situation where many of the risks may be difficult to foresee, let alone quantify, making AI risks an area of “deep uncertainty.” Despite this, some researchers have begun applying probabilistic methods to AI risks by describing possible adverse consequences of AI in coarse terms, such as “existential catastrophe.” This type of analysis may be useful for supporting strategic policy-level considerations, e.g. whether to impose a moratorium on training of large-language models, or whether to create an international body to regulate AI research. However, this high-level view if not particularly useful for engineering a given implementation of AI or ensuring its safety—e.g., improving the safety of self-driving cars, preventing consumers from using chatbots for health advice, or barring criminal groups from using AI advisers to help prepare explosives. Here, we argue that many practical AI problems could be addressed by drawing from a toolkit of non- probabilistic risk-management methods. Although less familiar than probabilistic methods, there exists a large class of strategies for managing risks that could be utilized in new AI-driven products. In particular, we argue that there are numerous non-probabilistic methods drawn from fields such as safety engineering, product management, and even military planning. These techniques could be used to better understand and anticipate AI risks, as well as for reducing or mitigating AI risks even when they are imperfectly understood or cannot be assigned probabilities with confidence (we list 70 such methods at https://github.com/sashagutfraind/uncertainty_strategies). Methods for anticipating AI risks range from fault trees and event trees at the most sophisticated, to hazard and operability studies or failure mode and effects analysis, to scenario analysis and robustness analysis. Simple but effective solutions for reducing risks include checklists, what-if thinking, and pre-deployment testing (including adversarial testing). Some of the latter methods are well-suited for use even by non-expert users. With regard to techniques for reducing or mitigating AI risks, we distinguish between strategies for safe design and strategies that enable rapid reaction to undesired behaviors. Included in the former list are fail- safe design principles (e.g., isolation of critical systems, or human-in-the-loop for critical decisions), prototype-driven development, and staggered rollout. Reactive solutions include contingency planning, monitoring and anomaly detection (either by AI or by human monitoring), and dedicated rapid response units. These strategies can operate in parallel, creating layered protection or “defense in depth” (possibly including the ability for humans to intervene) that can dramatically reduce the risk of undesired behaviors. We argue that drawing on this repertoire of non-probabilistic methods of risk analysis and risk mitigation should make it possible to develop safer AI applications, while still allowing the field to advance and reap the socio-economic benefits promised by advanced AI technology.

Type of Resource

text

Genre of Resource

Conference Paper/ Presentation

Language

eng

Handle URL

https://hdl.handle.net/2142/121857

Owning Collections