Very interesting, are there any more intuition on dense vs MoE? Also hate to be that guy.. but is there a public source for this (to test okay if not)
Dense vs Mixture of Experts: Technical Insights and Public Sources
By
–
By
–
Very interesting, are there any more intuition on dense vs MoE? Also hate to be that guy.. but is there a public source for this (to test okay if not)