The key challenge in AI development is ensuring that as systems become more capable, they remain aligned with human values. Traditional methods often force a trade-off—either high capability with alignment risks or strong alignment with limited capability. One possible solution is an approach inspired by AlphaGoZero, which iteratively improves AI through two complementary processes:

How Iterated Improvement Could Work

This approach involves repeating two key steps: amplification and distillation. First, a human overseer leverages multiple copies of the current AI model as subroutines to solve complex tasks beyond their individual capability. This "amplification" step combines human judgment with AI efficiency. Then, the system distills this amplified behavior into a new, faster model using safer, narrow learning methods that reduce misalignment risks. For example:

• A basic personal assistant AI could first learn by imitating a human.
• The human then uses multiple copies of this AI to handle more advanced tasks like scheduling and research.
• The improved behavior is distilled into a more capable assistant, repeating the cycle.

Advantages Over Existing Approaches

Unlike reinforcement learning, which risks reward hacking, or imitation learning, which limits growth, this iterative method allows gradual capability scaling while preserving alignment. Compared to similar frameworks like AlphaGoZero (which is game-specific) or cooperative inverse reinforcement learning (which lacks iterative improvement), it provides a more generalizable path to advanced AI assistance.

Getting Started With the Idea

A minimum viable test could involve applying this process to a constrained task like email management—training an initial model on basic sorting, amplifying its usefulness through human-AI collaboration, and distilling improvements into a refined version. Early-stage validation could examine whether distillation preserves alignment by comparing AI behavior against human expectations at each stage.

The approach presents a potentially scalable way to develop AI that matches both human values and complex needs, though real-world testing would be needed to verify its assumptions around preservation of alignment during iterations.