How to Write Forecasting Questions

Prediction markets are a niche topic. This post is for a niche within that niche: for the creators and writers of prediction markets.

Asking the right questions is necessary for good forecasting. While it could be considered one aspect of forecasting, prediction markets separate the writing and the predicting into different roles.

Made nearly two hundred markets myself so far and I development something like a style.

My Style

Kind of my template on Manifold looks like this:

1. Motivation in the form of a quote from a news article.

2. Data sources for resolution. Potentially with a naive forecast.

3. “Resolves YES if…”

4. “Resolves NO if…”

5. “Resolves NA if…”

6. Potentially, “Close date might be extended.”

Step 4 may sound redundant since it is just the opposite of 3, but it is also incredibly helpful sometimes, when people ask about edge cases in between.

Here is an example:

Will Spain have glaciers in 2033?

Rising temperatures forecast to wipe out Spain’s remaining glaciers in 10 years:

In 1850, when global temperatures began to rise steadily during the Industrial Revolution, there were 52 glaciers in the Pyrenees. By 2020, only 21 remained and almost 90% of their entire surface area has been lost.

Wikipedia claims "In 2006, ten small glaciers and six glaciers-glacierets remain in the Spanish Pyrenees."

Resolves NO if Spain is declared glacier-free by an institution like the Pyrenean Institute of Ecology (IPE).

Resolves YES if at least one glacier, like Aneto, still exists in August of 2033.

I’m aware that there is a gap in the resolution criteria: What if no existing glacier is there according to Wikipedia (hence not YES), but there is glacier-free declaration by the institute (hence not NO)? For now, I accept the risk to be a little ambiguous here.

There are also some variations. For example: “Resolves YES immediately if \<X happens>. Resolves NO by \<date>.” This is clear language that the close date will not be extended. The “immediately” makes clear it this can resolve early. Here is an example:

Will Stuttgart 21 finish before 2026?

Stuttgart 21 is railway project around Stuttgart main station which costs nearly 10 billion euro at this point. It converts the current terminus station into an underground station. The latest estimate for start of operation I found is December 2025 (initially 2019).

For this question, we define "start of operation" when they made an official announcement, a train stops at the underground station, passengers board or exit, and then the train departs to the other side (not a terminus station). Test runs do not count.

Resolves YES immediately if the start of operation happens.

Resolves NO on 2026-01-01.

There are also multiple choice questions with more answers than YES and NO. Manifold even offers the option to extend them later.

There is a tradeoff though. Binary questions are simple and easy. Here are some possibilities which are practically about the same question:

1. Will the Fed raise interest rates in the December meeting? (NO) A simple question which is essentially what most people cared about. There is no distinction between “hold current rates” and “lower rates” though.

2. Fed interest rate decision in January 31? (lower/hold/raise) Here we acknowledge that are actually three possibilities. There is no consideration how drastic they might change the rates though.

3. Fed interest rate decision on March 20? (lower by 0.25%/hold/raise by 0.25%/Other) The answer “Other” is for more drastic changes. If it rises from 3% to 10% I can split it into more answers. Maybe “lower by 0.5%” gets discussed in February.

There is no “best” approach and we could also do all in parallel.

Generalizing to Principles

You don’t want to copy my markets directly and something is unique about your markets. Let me try to generalize my style to actionable principles.

• Start with your motivation. The impulse to open a prediction market is often some news article, a tweet, or a discussion with friends. This is helpful because the concrete question is often only about some detail. (Example. My focus here is the impact of Russia’s behavior on world hunger and the wheat price in South Africa a proxy metric. That was helpful to clarify the currency question in the comments.)

• Provide estimation help. You probably have your own idea about the probability already, so do a little homework and add that to the question. This makes it easy for people to simply judge “a little higher” and get involved. (Example: In this market, I included a naive forecast in the description: “Extrapolating that to 2028 results in 34 trillion USD.”)

• Use terms which are is to understand for the title. (Example: Germany's GDP in 2023? There are multiple variant of “GDP” [real vs nominal, measured by whom, in what currency, etc] but for the title such detail is not necessary.)

• Use correct terms in the description. It helps to have some domain knowledge about who exactly is the actor and what it is called what they do. (Example. The title is about the “Fed” but the description is more precisely about the “FOMC meeting”.)

• Focus on short-term questions. Especially, if the platform is about betting (i.e. not Metaculus), short-term means higher return of investment for traders. (Example: From this open-ended question, I made a 2024 question)

• Avoid negations. Words like “not” are easy to overlook or misunderstand. Since questions are usually easy to invert, go for the version without negations. (Example: the “lose control” phrasing is negative. Simpler would have been: Will Hamas still control Gaza strip by April?)

• Keep it short. Long paragraphs turn people off. (Short example)

Metaculus now has “multiple choice questions”, which means multiple options which sum up to exactly 100%. Doing that manually was certainly annoying.

The latest drama on Manifold: @Mira rage-quitting and selling everything. She announced it on Twitter.

If a whale, like Mira, makes sudden moves a lot markets get rattled and people are confused about the sudden price movements. At the end of the year a lot of markets close and for many the outcome is already clear, so there is free mana around and the other whales had no cash ready to counter Mira immediately.

Enough drama now though. Tradition requires us to kill some trees and decorate their carcasses for celebration.