Load balancing, caching, dynamic batching… with 10% DAU you probably don’t need much more than 1k H100s to serve ChatGPT to 100 million customers.
@rasbt
-
AI as Brainstorming Aid for Medical Diagnosis Exploration
By
–
They don’t have to blindly rely on the results but could consider it as an additional brainstorming aide. I think the most interesting use case is something like: “given symptoms xyz, what could be the possible causes to further look into / follow-up with additional tests”
-
AI’s Potential Impact on Medical Practice and Doctor Adoption
By
–
And that’s the general purpose model. If you develop a finetuned version with and for doctors, it could have such a positive impact.
(Also, doctors not taking advantage of AI is kind of like doctors not consulting books; if used properly it doesn’t make sense not to) -
Deep Learning Jargon Explanations in Statistics Course Introduction
By
–
When I was a fresh statistics prof at UW-Madison & teaching Deep Learning class in the stats department, I had a bunch of “sorry for the DL jargon” slides about that in the intro lecture
-
The Matrix Cookbook Reference Guide
By
–
The Matrix Cookbook, https://
www2.imm.dtu.dk/pubdb/pubs/327
4-full.html
… -
Praise for Olmo Research Paper Over LLM Whitepapers
By
–
Nice and so well deserved! Loved the Olmo paper. It’s an actual research paper, not a whitepaper like those other LLM reports.
-
Quantization-aware training recommended for most techniques
By
–
Yes, for most techniques you’d want to do
Quantization-aware training. -
Quantization-Aware Training: Keeping Weights as Floats During Training
By
–
Yeah you can’t convert weights to int for training because you need to calculate the gradients during training. There are lots of different techniques for QAT, maybe this implementation helps: https://
pytorch.org/blog/quantizat
ion-aware-training/
… -
Claude’s Diagram Creation Capabilities Impress Developer
By
–
Wow this looks indeed spot on. Haven't used/tried Claude for diagrams, yet, and this makes me feel like I've been living under a rock (Creating the diagrams for my blogs & books is easily the most work)
-
Attention Mechanism Chapter: Challenging but Rewarding Learning Experience
By
–
Nice! The attention mechanism chapter might be the scariest one, but I think it gets more fun and rewarding after this one! (But lots of people told me that they esp. like the attention chapter…we will see )