Looking fwd to presenting a new talk: "Data is All You Need: Large Robot Action Models and Good Old Fashioned Engineering" @Stanford Robotics Seminar on Friday: https://
stanfordasl.github.io/robotics_semin
ar/
…
MACHINE LEARNING
-
Data is All You Need: Large Robot Action Models and Engineering
By
–
-
Dataset Limitations: Missing the Full Distribution of Correct Answers
By
–
Issue in mind is not so much human bias but the fact that the full distribution of correct or desirable answers to your prompts is almost certainly not present in your dataset, only a few samples.
-
The Ambiguity Problem in LLM Labeling and Training Data
By
–
Consider being a labeler for an LLM. The prompt is “give me a random number between 1 and 10”. What SFT & RM labels do you contribute? What does this do the network when trained on? In subtle way this problem is present in every prompt that does not have a single unique answer.
-
H2O.ai Named Leader in Forrester Wave Computer Vision Tools Q1 2024
By
–
http://
H2O.ai is a LEADER in The Forrester Wave: Computer Vision Tools, Q1 2024 We received the highest scores possible in across a wide variety of criteria including data and model testing, innovation, and more! Follow the link to learn more: https://
h2o.ai/platform/Forre
ster/
… -
LLMs Achieve Expert-Level Ophthalmological Knowledge and Reasoning
By
–
"Large language models (LLMs) are approaching expert-level ophthalmological knowledge and reasoning" https://
journals.plos.org/digitalhealth/
article?id=10.1371/journal.pdig.0000341
… @PLOSDigiHealth -
PyTorch DDP Gradient Syncing Bug in Distributed Training
By
–
so last week I was posting about a bug i had with gradient syncing in torch DistributedDataParallel (DDP), aka the most standard way to do multi-GPU training for neural networks all of my misery originated with a single design decision from the torch team: DDP shares
-
Five Years Between Transformer Attention and FlashAttention Innovation
By
–
"5 years between Self-Attention Is All You Need and FlashAttention"
quite incredible stat, gives a pause -
Modeling Different Coprocessors with Varying Instruction Sets
By
–
another layer of the onion 🙂 maybe can you model it like having different kinds of coprocessors with different instruction sets / reliability rates
-
Computing History Repeats: From Bytes to Tokens
By
–
The history of computing is repeating in an echo, except replace computers that do precise arithmetic on bytes with computers that do statistical arithmetic on tokens.
-
Fine-tuning Strategy: Sequential Layer Training Approach
By
–
Has anyone tried fine-tuning half of the layers in one run, and then the other half in another? cc @teknium @winglian @erhartford