Natural conversation includes interruptions and talking over people, which is hard for an LLM to model as a single autoregressive sequence. I’m sure you can get pretty far by creating a text sequence with movie-script like breaks mid sentence, but it seems like the real solution

→ View original post on X — @id_aa_carmack,