Arize Community Paper Reading: The Illusion of Thinking

​Join us as we discuss The Illusion of Thinking, a new paper from researchers at Apple that challenges today’s evaluation methods and introduces a new benchmark: synthetic puzzles with controllable complexity and clean logic. Their findings? Large Reasoning Models (LRMs) show surprising failure modes, including a complete collapse on high-complexity tasks and a decline in reasoning effort as problems get harder.

We'll dive into the paper's findings as well as the debate around it, including The Illusion of the Illusion of Thinking. It's sure to be a great discussion with some potentially spicy takes!