We use self-supervised learning to pretrain LLMs (e.g., next-word prediction). Here's an interesting take using self-supervised learning for evaluating LLMs: https://
arxiv.org/abs//2306.13651
Turns out, there's correlation between self-supervised evaluations & human evaluations.

→ View original post on X — @rasbt,