Accurate prediction aggregation is crucial for decision-making in fields like finance, science, and policy, but existing methods often assume experts share the same baseline beliefs. When this assumption fails—which is common—aggregated predictions become unreliable. One approach to address this gap could involve focusing on how experts update their predictions relative to a shared baseline, rather than treating their raw forecasts as directly comparable.

How the Method Works

Instead of averaging raw predictions, the idea suggests isolating each expert’s unique insight by measuring how much their forecast deviates from a baseline prior (e.g., historical event resolution rates). These deviations are combined in a weighted sum, with an optional extremization step to sharpen meaningful disagreements. For example:

• Baseline prior (e.g., historical accuracy or domain consensus) sets a reference point.
• Experts’ predictions are compared to this baseline as log-odds deviations.
• Deviations are aggregated and adjusted for confidence, yielding a final prediction.

Why It Matters

Forecasting platforms could benefit from this method by improving accuracy and transparency. Experts might appreciate a system that better reflects their skill, and end-users—such as policymakers—could rely on sharper aggregated forecasts. A lightweight implementation could start with synthetic data or small-scale tests on existing platforms like Metaculus, refining weights and extremization through experimentation.

This approach differs from traditional averaging by explicitly accounting for divergent expert priors. While challenges like noisy deviations or novel-event baselines exist, adaptive weighting and dynamic reference points might help mitigate them. If successful, it could complement existing aggregation methods without requiring major computational overhead.