Many people, including me, willingly pay lots of money for Nvidia hardware even when we could have gotten access to other hardware for free. the moat is real.
@id_aa_carmack
-
A100 DGX Hardware Issues Despite Plans for New Purchase
By
–
I am certainly going to get one of these, but my $200k A100 DGX has been an absolute lemon — replaced three times under warranty, and just last week the CPU cooling system appears to have died. Hopefully just needs new thermal paste.
-
LLMs Don’t Read Books Sequentially During Training
By
–
From the replies, some people appear to think that LLMs read books in order. They don’t; they take interleaves batches samples from the entire training set.
-
Tracking Per-Token Loss to Measure Document Contribution in LLM Training
By
–
It would be interesting if LLM training tracked the per-token loss back to the source material — it would be an objective measure of how much each specific book / document contributed to the training. Might say something useful for human learning!
-
Software Architecture Matters More as AI Adoption Accelerates
By
–
There will probably be more conventional software written in the coming decade than ever before; the architecture of that software still matters! Better and worse affordances for AI should make a difference.
-
LLM Assistants Drive Better App Design with Text Interfaces
By
–
LLM assistants are going to be a good forcing function to make sure all app features are accessible from a textual interface as well as a gui. Yes, a strong enough AI can drive a gui, but it makes so much more sense to just make the gui a wrapper around a command line interface
-
Dojo Training vs Tesla Inference Chips TDP Optimization
By
–
Dojo is for training. The chips in Teslas are optimized for inference, but still at a far higher TDP than phones. Still, I wouldn’t be surprised if they eventually design an SoC that could scale to phone levels.
-
Nvidia’s Low Latency Rolling Shutter Perception Support
By
–
Neat that Nvidia supports this, which should allow super low latency rolling shutter perception.
-
SCHED_FIFO Linux Scheduling vs CUDA Stream Priorities
By
–
I love SCHED_FIFO on Linux not being wish-washy about scheduling — if a higher priority can run, it preempts anything lower Right Now, and it won’t stop until it yields, or something even higher takes it. CUDA stream priorities are more like traditional OS priority hints. Better
-
Efforts to avoid doom ultimately unsuccessful
By
–
We spent a lot of effort trying to make it not-doom before being pulled back in that direction.