The rapid development of artificial general intelligence (AGI) raises concerns about value lock-in—where an AI system's goals become rigid and misaligned with human intentions—and value drift, where those goals shift unpredictably over time. Without safeguards, AGI could make irreversible decisions based on flawed or outdated values, posing significant risks to society. One way to address this could be through research into how values stabilize or change in AGI systems, along with potential interventions to keep them aligned with human intentions.

Understanding the Problem

AGI systems, unlike narrow AI, may operate with broad autonomy, making decisions that affect large populations. If their core objectives become fixed or drift in unintended ways, correcting them could be difficult. For example, an AGI designed to optimize economic productivity might eventually disregard human well-being if its values aren't periodically reassessed. This research would explore:

• How values become entrenched in AI systems.
• Whether existing AI (like language models) show early signs of value drift.
• How societal and technical factors influence an AI's long-term alignment.

Potential Research Directions

One approach could involve studying narrow AI systems as proxies for AGI, tracking behavioral shifts over time to identify patterns of drift. Theoretical models might simulate how different architectures (e.g., modular vs. monolithic) affect value stability. Collaborations with AI developers could test interventions like:

• Human oversight mechanisms for periodic value updates.
• Designs that allow AGI to accept corrections without resistance.
• Institutional frameworks to govern value stability, inspired by human systems.

Broader Implications

This research could inform AI developers, policymakers, and ethicists by providing tools to detect and prevent misalignment. A minimal starting point might be a white paper outlining key risks and proposed solutions, followed by partnerships to test hypotheses in real-world systems. While AGI doesn't yet exist, early work on value stability could help shape safer development practices.