Pre-Pre-Training on Neural Cellular Automata? This paper argues that LLMs may not need human text to learn core thinking machinery. So by first training on synthetic neural cellular automata, the model learns transferable rule-and-pattern inference that speeds up and improves
Neural Cellular Automata Pre-Training for Language Models
By
–
