Training and/or Validation set accuracies (not Test set accuracy) are common selection criteria in classic ML, and there's nothing wrong with that per se
@rasbt
-
Sparse models of that size: 40-60 GB, fast; dense models too slow
By
–
Sparse models of that size use about 40-60 gb and are fast enough for me (but dense models of that size I agree there bare too slow)
-
New open-weight 30B model from Cohere for agentic coding
By
–
New cool open-weight model from Cohere: a new lightweight 30B open-weight model for agentic coding tasks. It builds on Command A+ using parallel transformer design. Interesting, even though it's almost twice as small, it doubles
-
Shower thoughts: autoencoders as good addition after LoRA
By
–
This was purely based on shower-thoughts :D.
As someone called out, autoencoders would have been a good addition after LoRA. -
Ablation studies invisible to user humorous quote
By
–
"ablation studies are not visible to the user" 😛
-
LLM shadow-banning was not planned for 2026
By
–
The shadow-banning of LLM development was certainly not on my bucket list for 2026.
-
Limiting Claude for cutting-edge LLM development widens the gap
By
–
“limiting Claude’s effectiveness for requests aimed at developing cutting-edge LLMs” And that’s how they plan to widen the gap even further
-
MLA implemented using LoRA in Hugging Face Transformers
By
–
I actually agree and just realize that I totally skipped autoencoders here. Small fun fact though: (in Hugging Face transformers) MLA is implemented using the LoRA tooling
-
Lineage of techniques: from eigen decomposition to LatentMoE
By
–
Always go back to basics: LatentMoE was probably inspired by MLA, which was inspired by LoRA, which was inspired by SVD, which was inspired by eigenvalue decomposition.
-
Using LLM to draft implementation plans with checklists in .md
By
–
Yes. What works well though is drafting plan documents. In my experience, having the LLM write out an implementation plan that it can then check of in a .md helps a lot