I am having discussions with them — there are some straightforward pixel quality improvements that can be made, but the big questions about depth estimation and inpainting trade offs are contentious.
@id_aa_carmack
-
Reading Nanosystems and reconsidering Superintelligence
By
–
I have been meaning to read Nanosystems — I just ordered a copy. I did read Superintelligence, both before I started on AI, and again this year, but I didn’t find it compelling either time.
-
Carmack focuses on research beyond language models
By
–
I am not working on anything language model based; there are plenty of people doing that. All research for the foreseeable future.
-
DGX Station Hardware Failures and Support Experience
By
–
On the 4-GPU DGX Station, I had a general liquid cooling system failure, a CPU cooling failure, and a GPU failure. Support was great, and promptly replaced the system each time, but it still sucks.
-
GPU cluster reliability issues exceed conventional systems by orders of magnitude
By
–
I’m sure my thrice-replaced DGX station is an outlier, but the reliability reports I hear from big GPU cluster people are still grim. They have over an order of magnitude more failures than conventional systems. ML training is tolerant, but it is at a point that gets noticed.
-
H100 GPUs lack gaming hardware, potential for CAVE rendering demo
By
–
The H100 GPUs don’t have either rasterization or ray tracing hardware, so conventional game engines can’t run on them. I do think it would be a fun to do some kind of super high end 240 fps CAVE rendering demo with a cluster!
-
Open Source and Program Scale Reuse as Software Development Silver Bullet
By
–
I do think that open source and program scale reuse (vs function scale) turned out to be the closest thing to a silver bullet we got.
-
Why Tech Giants Don’t Train ML Models Longer Despite Low Costs
By
–
It has been pointed out that even $100M ML model training costs aren’t that large in tech titan terms, so I wonder why they don’t just keep training them for far longer just to see what happens. Overfitting is a concern, but you might also hit a grokking discontinuity.
-
GPU Power Consumption: High Current Flow at Low Voltage
By
–
I didn’t really appreciate that, because they operate at under one volt, high end GPUs are continuously flowing over 500 amps through the die at full load.
-
Numeric representation for efficient fused multiply-add hardware
By
–
Conventional numbers are easy to add/subtract and logs are easy to multiply/divide — I wonder if there is a numeric representation that could support more efficient hardware fused multiply-adds for matrix multiplies.