Managing it at that scale sounds like a challenge though. I was a moderator for the machine learning category (cs.LG) on arxiv for a few years: we got like 100-300 paper submissions per day! Just checking whether the categorization was correct already took 1-2 hours each day.
@rasbt
-
CPU Performance Progress: Fast MLP Training on Modern MacBooks
By
–
I actually recently marveled at how far CPU performance has come (comparing my undergrad computer to a modern MacBook). Was creating a confidence interval demo with a simple sklearn MLP, and it fits 500 iters in like a second on a 2020 MacBook Air: https://
github.com/rasbt/MachineL
earning-QandAI-book/blob/main/supplementary/q25_confidence-intervals/1_four-methods.ipynb
… -
Using ArXiv Carefully: Value Despite Imperfections
By
–
So much discussion about arxiv this morning. You can get a lot of positive value out of using arxiv if you use it with care. Eg does the original Transformer paper have inconsistencies (see LayerNorm) and is not super detailed? Sure. Is it a groundbreaking paper. Yes!
-
Citation Context Matters: Different Outcomes Change Architecture Proposals
By
–
Depends on the cites. E.g.,
“xyz proposed a new architecture that successfully addresses …” is quite different from “xyz proposed a new architecture that failed to address …” -
Training Parameters vs Sample Size: Statisticians’ Historical Concerns
By
–
I think it wasn’t that computers were to slow, but rather getting yelled at by statisticians if the number of training parameters was bigger than the sample (training dataset) size
-
T5 and Flan-T5 Models Expected to Remain Highly Relevant
By
–
I'd say T5 and Flan-T5 are probably going to stay very relevant!
-
BERT and RoBERTa remain top models for classification tasks
By
–
BERT/RoBERTa are still among the best models for classification tasks. Even Twitter uses them here for their platform.
-
Distilled Models Remain Useful: GPT-3.5 and GPT-4 Examples
By
–
I do think that distilled models are still useful. I.e., GPT 3.5 in ChatGPT is a distilled model. And as far as I heard GPT 4.0 in ChatGPT is now also distilled.
-
PyTorch Distributed Training and Tensor Sharding with Fabric
By
–
None of these: just PyTorch with distributed training and tensor sharding. Optionally with CPU offloading for really big LLMs. I use Fabric as a convenient wrapper here.
-
AI Model Performance Less Impressive Than Data Memorization
By
–
Turns out, it’s even less impressive than memorizing the training data: