yes, it’s arithmetic coding using the LLM probability distribution over training token sequences
Arithmetic Coding with LLM Probability Distributions Explained
By
–
By
–
yes, it’s arithmetic coding using the LLM probability distribution over training token sequences