Insightful thread by @hwchung27 that shares some behind the scenes of training flan-t5. These details are often overlooked and sometimes mentioned in footnotes in papers (or not mentioned at all, i.e., "not paper worthy"). Sometimes they are stack dependent, model dependent or

→ View original post on X — @yitayml,