Maybe a container issue. I know Nvidia recommends their docker containers, but I am usually just running it on the Spark directly.
@rasbt
-
Working fine with Ollama and llama.cpp for Nemotron model
By
–
Weird, what issue are you having. Works for me both in ollama and native llama.cpp I am using unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF:MXFP4_MOE
-
Building Large Language Models From Scratch YouTube Series
By
–
I actually have a Build an LLM From Scratch YouTube series
-
Token Generation Speed Comparison: OSS and Nemotron Models Performance
By
–
Yeah. Not quite as bad for me, but:
gpt-oss-120b: 42.22 tokens/sec
nemotron-3-super: 20.43 tokens/sec on the DGX Spark. But this is ollama and might be an implementation issue. I have yet to try the Nvidia-optimized llama.cpp version (they had one for Nemotron Nano back then) -
Nemotron 3 Nano Performance with Model Already Satisfactory
By
–
Yeah. Although, Nemotron 3 Nano already worked quite well with it, so not the biggest surprise 🙂
-
Qwen3.5 Ranks 7th on Pinchbench as Impressive Small Open-Weight Model
By
–
Qwen3.5 is ranked at an impressive 7th place on pinchbench atm. It's really impressive for such a relatively small open-weight model.
I think that's a perfect use case for local LLMs.
Curious to see where Nemotron 3 Super will rank. -
Nvidia Nemotron 3 Super 120B: High Performance Open-Weight LLM
By
–
Another week, another noteworthy open-weight LLM release. Nvidia’s Nemotron 3 Super 120B-A12B looks pretty good. Benchmarks are on par with Qwen3.5 122B and GPT-OSS 120B, but the throughput is great! Below is a short, visual architecture rundown.
-
Qwen3.5 Implementation Integration Guide for LLMs
By
–
Thanks! I think it shouldn't be too hard. I have a Qwen3.5 from-scratch implementation here: https://
github.com/rasbt/LLMs-fro
m-scratch/blob/main/ch05/16_qwen3.5/qwen3.5-plus-kv-cache.ipynb
… that you can use as a drop-in replacement. Perhaps the easiest thing to do here is to take that and put it into the qwen. py (
https://
github.com/rasbt/LLMs-fro
m-scratch/blob/main/pkg/llms_from_scratch/qwen3.py
…). If you -
Chapter 8 LLM Distillation Notebook Released on GitHub
By
–
The Ch08 Nb on distilling LLMs is now on GitHub: https://
github.com/rasbt/reasonin
g-from-scratch/blob/main/ch08/01_main-chapter-code/ch08_main.ipynb
… Hard distillation that works with any LLM (minding the terms of service, of course).