REVIEW 1 cited by
Alignment Problems With Current Forecasting Platforms
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Alignment Problems With Current Forecasting Platforms
read the original abstract
We present alignment problems in current forecasting platforms, such as Good Judgment Open, CSET-Foretell or Metaculus. We classify those problems as either reward specification problems or principal-agent problems, and we propose solutions. For instance, the scoring rule used by Good Judgment Open is not proper, and Metaculus tournaments disincentivize sharing information and incentivize distorting one's true probabilities to maximize the chances of placing in the top few positions which earn a monetary reward. We also point out some partial similarities between the problem of aligning forecasters and the problem of aligning artificial intelligence systems.
Forward citations
Cited by 1 Pith paper
-
Global Merger-Arbitrage Forecasting with Language Models
Expert-context research agents plus hindsight-guided finetuning cut class-balanced Brier score on merger outcomes to 0.151, beating calibrated market prices, XGBoost, and frontier LLMs.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.