It is easy enough to make your own, but I think standard relu should have been defined as passing the value at zero, so gradients flow backward through it, allowing some things to be zero weight initialized when symmetry breaking isn’t an issue.
@id_aa_carmack
-
Grok AI Feature Available for Premium Subscribers
By
–
Pretty sure that was about the SNES Wolfenstein port. I had to stop working on Doom and do that myself after the contractor completely flaked on us.
-
Anita Sarkeesian: Media Critic and Feminist Frequency Founder
By
–
He might have been conflating the story about Bill Heineman and the SNES Wolfenstein port, which was around that time.
-
CUDA Memory Optimization: 511×512 vs 512×512 Performance Analysis
By
–
Some people are misreading this — 511×511 was FASTER. It looks like at 512×512 and above it falls to another path that requires internal CudaMalloc/Free calls.
-
GPU Matrix Computation Performance Drops at 512×512 Threshold
By
–
GPU library performance can be very notchy — runtime of batched torch.linalg.solve_ex() went up by over 10x going from 511×511 matrices to 512×512.
-
AI-Powered Elevator System Optimization on Modern Cruise Ships
By
–
I was on a cruise ship last week (Star of the Seas), and they had pods of 10 elevators in a circle, where you picked your destination floor on a pad, and it directed you to the correct elevator, which was often behind you.
— John Carmack (@ID_AA_Carmack) 27 avril 2026
It seemed to work efficiently, but multiple times I saw… pic.twitter.com/NGPgENGrqlI was on a cruise ship last week (Star of the Seas), and they had pods of 10 elevators in a circle, where you picked your destination floor on a pad, and it directed you to the correct elevator, which was often behind you. It seemed to work efficiently, but multiple times I saw
-
LLM Training for Lossless Data Compression Internet Archive
By
–
It is generally frowned upon to have LLMs precisely regurgitate part of their training set, but it is an interesting question how you could use LLM training to nearly losslesly compress a huge corpus like the entirety of the Internet Archive. The Hutter Prize is for perfect
-
A Canticle For Leibowitz: Nuclear Holocaust and Knowledge Preservation
By
–
A Canticle For Leibowitz is a classic early (1959) post-apocalypse novel where an order of monks preserved the last remnants of learning (the memorabilia) after a nuclear exchange turned the remains of society into book and scientist burners. I first read it in the 80s as a mass
-
FLOPS: From Rate Measurement to Algorithm Work Quantification
By
–
FLOPS was originally “floating point operations per second”, specifying a rate of work for a system: A SPARCstation 2 gave 4.2 MFLOPS. Today you also see it used as “floating point operations” for an algorithm, or an amount of work: This layer takes 8 GFLOPS.
→ View original post on X — @id_aa_carmack, 2026-04-13 18:35 UTC
-
Carmack’s Law Prevails: Computing Power Continues Exponential Growth
By
–
Rhymes with @RichardSSutton’s Bitter Lesson. Tim Sweeney (@TimSweeneyEpic) A computer can do anything provided you learn to tell it how. Very recently, this has become vastly easier to do. Chalk up another victory for Carmack’s Law: — https://nitter.net/TimSweeneyEpic/status/2043434467339931651#m
→ View original post on X — @id_aa_carmack, 2026-04-13 16:18 UTC