Yes, sorry I see what you mean. I didn’t mean to imply that they studied these longer-context models like LongNet above. I was just referencing that we talked about these models (it would have been impossible for the authors to eval it at that time, but hopefully in the future)

→ View original post on X — @rasbt,