You can encode a grammar into a transformer. The problem is gradient descent is not very good at learning it.
Encoding Grammar in Transformers: Gradient Descent Learning Challenges
By
–
By
–
You can encode a grammar into a transformer. The problem is gradient descent is not very good at learning it.