The Brain-Score Benchmarking Competition aims to find behavioral and neural benchmarks on which the predictions of our current top models break down.

We are entering the final week of the 2024 competition – please submit your benchmarks to highlight the flaws in current state-of-the-art models of vision (and win prizes)!

In 2022, the first Brain-Score Competition led to new and improved models of primate vision that predicted existing benchmarks reasonably well. In this year's Brain-Score Competition 2024, we are closing the loop on testing model predictions by rewarding those benchmarks that show where models are the least aligned to the primate visual ventral stream. The Competition is open to the scientific community and we provide an infrastructure to evaluate a variety of models on new behavioral and neural benchmarks in a standardized and unified manner. In addition, we will incentivize benchmark submissions by providing visibility to participants and a $10,000 prize pool to the winning benchmarks.

Submissions are open until June 30, 2024. For regular updates related to the competition, please follow Brain-Score on Twitter and join our community. Good Luck!

Competition Tracks

• Behavioral: This track will reward those benchmarks that show-case model short-comings in predicting behavior (human or non-human primate). The winning submissions will be the behavioral benchmarks with the lowest model scores, mean-averaged over all models, i.e. as close as possible to 0. The benchmarks can target any behavioral task that the models engage on (labeling, class probabilities, odd-one-out ; see the model interface).
  
  1^st prize: $3,000; 2^nd prize: $2,000.
  
  
• Neural: This track will reward those benchmarks that show-case model short-comings in predicting neural activity across the primate visual ventral stream. The winning submissions will be the neural benchmarks with the lowest model scores, mean-averaged over all models, i.e. as close as possible to 0. The benchmarks can target any region(s) in the visual ventral stream: V1, V2, V4, IT (see the model interface). Brain recordings can be from human or non-human primates.
  
  1^st prize: $3,000; 2^nd prize: $2,000.