I'm teaching a little workshop this semester at cornell tech called Practical Deep Learning! it's supposed to be an intro to training neural networks and debugging them, in a modern context this is the schedule for the course. will post content here: http://
jxmo.io/deep-learning-
workshop/
…
@jxmnop
-
Practical Deep Learning Workshop at Cornell Tech
By
–
-
GPT-4 Turbo Dominates Language Models Like Magnus Carlsen
By
–
gpt-4 turbo is the magnus carlsen of language models (highest ELO)
-
Chatbot Arena: Language Models Racing for Global Champion Status
By
–
chatbot arena is a really fun measure of progress. it's exciting to see new models pop up with surprisingly high ELO scores language model development is turning into international chess, with a public leaderboard and a very clear global champion at all times
-
ROPE and Positional Embeddings: Sine Cosine Functions
By
–
i guess ROPE and some positional embeddings use sine and cosine as well
-
Understanding Transformers with High School Mathematics
By
–
one exciting observation about transformers (and most modern deep learning) is that you can understand them using high school math. really just multiplication, division, sums, and exponentiation, many times, and in a strange and initially hard-to-grok order
-
GPT-4 Turbo outperforms GPT-4: surprising benchmark results analyzed
By
–
surprised to see so many people excited to see google sitting in second place on a leaderboard also, the obvious question here is why is GPT-4 turbo beating GPT-4 on this benchmark? i thought turbo was intended to be faster but slightly dumber
-
Running Data Through Neural Network Multiple Times
By
–
thanks! but there’s no data loader here, and I’m not doing training; just running the same data through the network many times
-
ResNet Memory Bound vs Compute Bound Performance Analysis
By
–
i think it's something weird about resnet being memory bound instead of compute bound
-
PyTorch Model Inference Optimization with torch.no_grad()
By
–
this isn't the case bc i'm using the same image every time; my code looks like this for _ in range(n_iters): with http://
torch.no_grad(): model.forward(x) -
GPU Inference Batch Size Scaling: Why Limited Speedup?
By
–
when doing inference on lots of samples, how much speed up can you expect from increasing the batch size? context: i'm doing inference for lots of images using resnet100 on an a6000 gpu. increased batch size 32 -> 2048 (64x!) and only getting a 20% speedup. how is this possible?