Methodology
Overview
AI Prognostications tracks forward-looking, testable claims about AI's effects on the economy and society. We extract these claims from public articles, blog posts, and commentary, then catalog them with structured metadata to enable tracking over time.
What Counts as a Claim
We focus on claims that meet the following criteria:
- AI-specific: AI is the causal mechanism, not a tangential mention.
- Forward-looking: The claim makes a prediction about the future, not a description of the present.
- Time-bounded: The claim has an explicit or implied temporal anchor.
- Testable: The claim is specific enough to be confirmed or refuted with empirical evidence.
We exclude definitions, tautologies, citations of others' work, purely conceptual claims, and open-ended predictions with no time horizon.
How We Extract Claims
Articles are analyzed using a large language model (Claude) with a structured extraction prompt. The model identifies claims, tags them along six dimensions, and extracts verbatim quotes, timeframes, and confidence levels.
Claim Properties
Each claim is tagged along these dimensions:
- Economy
- Whether the claim concerns AI's effects on the economy (employment, GDP, productivity, wages, etc.).
- Society
- Whether the claim concerns AI's effects on society (education, governance, culture, inequality, etc.), independent of economic effects.
- Prediction
- Whether the claim is a prediction about future events or outcomes.
- Testable
- Whether the claim is specific and potentially confirmable or refutable with empirical evidence.
- Causal
- Whether the claim asserts a causal relationship (e.g., "AI will cause X").
- Comparative
- Whether the claim involves a comparison (e.g., "AI will increase X" or "more than Y").
Timeframes and Confidence
Where available, we extract the predicted timeframe for each claim and note whether it was explicitly stated or implied from context. We also capture stated confidence levels, whether expressed qualitatively ("highly likely") or as numeric probabilities.
Editorial Review
All extracted claims are manually reviewed before publication. Automated extraction can produce errors, and we check each claim for accuracy, relevance, and proper attribution before adding it to the public tracker.
Limitations
- Claims are extracted by an LLM, which may misinterpret nuance, context, or sarcasm.
- Timeframe and confidence extraction from free text is inherently imprecise.
- We track claims as stated; we do not assess their likelihood or correctness at the time of extraction.
- Our source coverage is not comprehensive and may reflect selection bias.