Researchers face at least six challenges in developing safe, human-aligned superintelligence (SI). First, we need safe SI by design. Second, we need a transparent and understandable SI. Third, we need to maintain some level of control as SI outstrips human ability to monitor its behavior. Fourth, we need means to align SI initially and maintain alignment as SI increases in intelligence. Fifth, we need scalable safety mechanisms. Sixth, the design for SI must handle potential exponential changes in the SI’s level of intelligence. The current approach to using machine learning to develop opaque models, supplemented by RLHF to test in safety, cannot meet these challenges. We need a new approach emphasizing safety and alignment by design. This paper presents a novel design for SI, leveraging the collective intelligence of many human and AI agents using a rigorous, transparent architecture that supports problem-solving, learning, and self-improvement. The design is compatible with current LLMs and foundation models. It is less costly, more powerful, and faster to develop than training trillion-parameter LLMs. Most importantly, it maximizes alignment with broadly representative human values and maintains dynamic alignment even as the SI surpasses human monitoring capabilities.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Designing Safe SuperIntelligence

  • Craig A. Kaplan

摘要

Researchers face at least six challenges in developing safe, human-aligned superintelligence (SI). First, we need safe SI by design. Second, we need a transparent and understandable SI. Third, we need to maintain some level of control as SI outstrips human ability to monitor its behavior. Fourth, we need means to align SI initially and maintain alignment as SI increases in intelligence. Fifth, we need scalable safety mechanisms. Sixth, the design for SI must handle potential exponential changes in the SI’s level of intelligence. The current approach to using machine learning to develop opaque models, supplemented by RLHF to test in safety, cannot meet these challenges. We need a new approach emphasizing safety and alignment by design. This paper presents a novel design for SI, leveraging the collective intelligence of many human and AI agents using a rigorous, transparent architecture that supports problem-solving, learning, and self-improvement. The design is compatible with current LLMs and foundation models. It is less costly, more powerful, and faster to develop than training trillion-parameter LLMs. Most importantly, it maximizes alignment with broadly representative human values and maintains dynamic alignment even as the SI surpasses human monitoring capabilities.