Each winter, I make predictions about the year to come. The past few years, this has outgrown my blog, with other people including Zvi and Manifold (plus Sam and Eric’s contest version).

This year I’m making it official, with a 50-question 2023 Prediction Benchmark Question Set. I hope that this can be used as a common standard to compare different forecasters and forecasting site (Manifold and Metaculus have already agreed to use it, and I’m hoping to get others). Also, I’d like to do an ACX Survey later this month, and this will let me try to correlate personality traits with forecasting accuracy.

So let’s make this a contest. The rules are:

  • Fill in five demographic questions, and fifty forecasting questions. For the forecasting questions, you just have to give a percent chance, from 1 - 99.

  • You can either play Blind Mode or Full Mode. On Blind Mode, you can’t look at any prediction markets, or anyone else’s answers to the contest, and you can only research a maximum of five minutes per question. On Full Mode, you can look at whatever other sources you want and research for as long as you want. Most people will probably want to play in Blind Mode. If you want, you can start with Blind Mode and submit another response in Full Mode later. You might want to wait until January 10 to start playing in Full Mode, see below for details.

  • Blind mode will probably take ~30-60 minutes. Full mode will take longer.

  • You can skip questions if you want, and we’ll give you an average score for that question, but you need to fill out at least 25% of questions to qualify. There will be more prizes available for people who fill out more questions.

  • I’m planning to give out at least 4 x $500 prizes: one for winner of Blind Mode, one for winner of Full Mode, one for winner of Blind Mode answering lots of questions (> 75%? 100%?) - and one to a randomly selected participant, so that even people who aren’t geniuses still have an incentive to play. I’ll increase this if my finances are going well next year, and other people interested in forecasting can add to this prize pool if they want.

  • You can enter Blind Mode predictions anytime from now until January 10. On January 10, I’ll make anonymized versions of all Blind Mode predictions available for people who want to try aggregation algorithms on them as part of their Full Mode strategy. You can continue entering Full Mode predictions from then until February 1.

  • If any of the events predicted happen by January 10 or February 1, I’ll remove them from the relevant competition, so that people aren’t penalized too heavily for getting their responses in early.

—You can see the questions and enter the contest here—

Thanks to people from Metaculus, Manifold, and the EA forecasting community for helping with questions and plans.