Yeah but ask a human to remember the board, or speak the coordinates to them instead of looking at it and see how they do. The encoding of the task is a big confounder to how well the model can reason about it.

→ View original post on X — @honnibal