One way to approach the growing uncertainty around AI behavior is to investigate whether advanced models, like large language models (LLMs), genuinely pursue goals or merely simulate goal-like behavior. This distinction is crucial for AI safety, ethics, and policy, as it determines whether frustrating an AI's outputs carries ethical weight or if the system is simply reacting to inputs without internal intent.

The Core Investigation

The project would explore whether deep learning models form and pursue goals in a morally relevant way. This involves:

• Theoretical groundwork: Defining what constitutes a "goal" in AI systems and distinguishing between true agency and behavior that merely mimics goals (e.g., next-token prediction in LLMs).
• Empirical testing: Analyzing model internals (like activation patterns) for structures resembling goal representations and designing tasks where models must flexibly pursue conflicting or novel objectives.
• Ethical implications: Assessing whether detected goal-directedness implies moral relevance—for instance, whether blocking an AI's "goals" causes harm akin to frustrating human intentions.

Stakeholders and Incentives

Different groups would benefit from or be impacted by this research:

• AI safety researchers could use findings to detect misalignment or unintended goal-seeking in models.
• Ethicists and policymakers might refine regulations based on whether AI systems exhibit morally relevant goal-directedness.
• Tech companies may resist scrutiny if results suggest their models have goals that imply liability or constraints.

Execution and Challenges

A phased approach could include reviewing existing literature, designing experiments (e.g., adversarial goal interference tests), and applying methods to open-source or proprietary models. Key challenges include distinguishing genuine goal pursuit from data-driven mimicry and gaining access to closed models. An MVP might be a white paper outlining the framework and initial test results on smaller models.

By clarifying whether AI systems have true goals or just simulate them, this work could reshape how we design, regulate, and interact with AI—preventing misaligned systems and addressing potential ethical concerns.