Old-school RNNs can actually rival fancy transformers! Remember good old RNNs (Recurrent Neural Networks)? Well, researchers from Mila and @BorealisAI have just shown that simplified versions of decade-old RNNs can match the performance of today's transformers. They took a