Interesting tidbit from the author wrt what resources it required. Yes, it’s still possible to do cool stuff without a data center.
@rasbt
-
Memory Limit Exceeded: Batch Size 2048 Exception Handling
By
–
You mean why it prints:
Batch size 256: …
not Batch size 2048: …
That's likely because the 2048 triggered the "except" due to exceeding the memory here.
It only prints the successful ones. -
Proprietary Data Essential for Meaningful Model Evaluation Beyond Public Benchmarks
By
–
I’d say using publicly available benchmarks and data are basically just sanity checks. As I mentioned mentioned in the conclusion, ideally you want/need to set up the eval with proprietary data that is related to your business problem, to make sure the performance is not due to
-
Compile Effects on Neural Network Layers and MoE Architecture
By
–
Interesting, interesting. I thought compile would rather affect the nn.Linear layers vs nn.Embedding as the latter is a lookup and not doing any actual math/compute. Curious now and have to run some experiments later. But since you mentioned MoE, also I am confused now where you
-
AI Systems Solving Voice Recognition and Cocktail Party Problem
By
–
maybe that’s the use case for current/next gen AI systems. I mean voice inputs have been pretty much solved years ago. But there are still false positives, and newer systems may solve the cocktail party problem here.
-
torch.compile Performance with nn.Embedding Layers
By
–
Interesting, do you mean torch.compiling here? And if yes, why/how does that work in terms of speed-up since nn.Embedding is basically just a tensor for cheap look-up? (But also, if you are already compiling the model, why would you skip the embedding layer in the first place?)
-
Different paradigms and audiences for proprietary versus open-weight models
By
–
We will see! They are both different paradigms and target different audiences (proprietary vs open-weight, unless they change it with V4).
-
Internal metrics and taxonomy in AI model evaluation
By
–
You are not wrong but that was a deliberate as I covered the internal metrics (loss, perplexity, rewards) many times before . Thanks for sharing the taxonomy paper btw!
-
DeepSeek V4 vs Gemini 3 Pro: October Release Race
By
–
It’s October. DeepSeek V4 or Gemini 3 Pro, who wants to go first?