Interestingly, DeBERTa-1.5B (and encoder-only model) beats Llama 2 on BoolQ, which is a nice example that encoders still outperform large decoders on classification task. For fairness: The DeBERTa-1.5B model was likely finetuned on the training data 1/3

→ View original post on X — @rasbt,