No, that's my code. But I used LLMs to help expand unit tests and run benchmarks outside the book etc.
@rasbt
-
Impressions initiales positives, mais performance ralentie avec llama.cpp
By
–
I have a good first impression, but it's still a tad slow for me (using llama.cpp). About 2x slower than gpt-oss 120B on the same hardware. I think I need to look for the NVIDIA-optimized stack.
-
TPUs vs Groq LPU: Training-Inference Trade-offs in Hardware Design
By
–
I agree with you. But I think TPUs are still trying to do a training-inference trade-off. I think Groq LPU would be an example that is purely designed for inference.
-
Trade-offs between training throughput and inference efficiency in chips
By
–
I think it's about trade-offs. As with so many things, there's no free lunch. Ie I don't think you can have a chip that is sota for training throughput but also sota in terms of inference efficiency at the same time
That being said, there are companies focused on inference chips -
Container Issues with Nvidia Docker and Spark Setup
By
–
Maybe a container issue. I know Nvidia recommends their docker containers, but I am usually just running it on the Spark directly.
-
Working fine with Ollama and llama.cpp for Nemotron model
By
–
Weird, what issue are you having. Works for me both in ollama and native llama.cpp I am using unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF:MXFP4_MOE
-
Building Large Language Models From Scratch YouTube Series
By
–
I actually have a Build an LLM From Scratch YouTube series
-
Token Generation Speed Comparison: OSS and Nemotron Models Performance
By
–
Yeah. Not quite as bad for me, but:
gpt-oss-120b: 42.22 tokens/sec
nemotron-3-super: 20.43 tokens/sec on the DGX Spark. But this is ollama and might be an implementation issue. I have yet to try the Nvidia-optimized llama.cpp version (they had one for Nemotron Nano back then) -
Nemotron 3 Nano Performance with Model Already Satisfactory
By
–
Yeah. Although, Nemotron 3 Nano already worked quite well with it, so not the biggest surprise 🙂