Now given that next-word prediction is multi-task learning, we can write the overall loss is the weighted sum of loss of individual tasks. When overall loss improves smoothly, do all individual tasks improve smoothly, or do some improve at different rates than others?

→ View original post on X — @_jasonwei,