Physics-informed learning is essential for AI+Science. Check out our latest paper in @PNASNews special edition on AI+Chemistry on incorporating simple rules of physics that two atoms in a molecule cannot get too close to one another (due to repulsive forces) vastly improves
MACHINE LEARNING
-
Complexity vs Feature Engineering Trade-offs in Machine Learning
By
–
The less complexity the more capacity to focus on learning other things. But the more handcrafted features the less opportunity to learn better representations. It all kind of depends on how big the model is, how much data you have, and what your compute resources are
-
New Tokenization Approach Challenges BPE-Style Methods
By
–
TLDR: surprised that it works, skeptical whether this will replace BPE-style tokenizers, delighted about this refreshing approach
-
Inference scaling costs exceed RL training for large-scale deployments
By
–
Depends. If you are serving millions of customers this type of inference scaling is going to cost you more than training with RL.
-
Image Format Complexity Requires More Training Data
By
–
exactly, that’s the messiness of working with the image format I mentioned. I think you can make to generalize well on all these but since there are more degrees of freedom it will require more data to train (luckily this can be done with automatic data augmentation but still)
-
How Constraints Drive Innovation in Neural Network Architecture
By
–
Yeah, I think that’s what brought us CNNS (as opposed to fully connected neural nets), LoRA, and many more
-
Reconsidering VLLM terminology at 3B parameters
By
–
Actually I was thinking about it when typing, and I don't know. I don't want to be that person who goes against the common terminology (like softargmax haha) but it really is a V*L*LM at 3B parameters.
-
DeepSeek OCR Paper and Multimodal LLM Article Resources
By
–
Link to the paper: https://
github.com/deepseek-ai/De
epSeek-OCR/blob/main/DeepSeek_OCR_paper.pdf
… My "Understanding Multimodal LLMs" article with more info on how images are fed to LLMs, how cross-attention works, etc: https://
magazine.sebastianraschka.com/p/understandin
g-multimodal-llms?utm_source=publication-search
… -
Visual Tokenization Challenges: Aspect Ratios and Image Preprocessing Complexity
By
–
I know it’s popular to hate tokenizers, but visual representations (which are also tokenized) bring a lot of messiness as well. Aspect ratios, cropping, resolution, brightness, etc. Sure, models learn to deal with that but it requires lots of data to make them robust wrt these.