LLMS

Transformers Outperform MLPs for Parsing Tasks

By

–

21 November 2022 4h08

Transformers are better than MLPs far from the limit. And these papers are about something else. All it takes for my statement to be correct is that a transformer can learn to parse correctly in some cases.

Transformers Represent Grammar Better Than MLPs

By

–

21 November 2022 3h50

No, transformers can represent grammar better than (say) MLPs, which is an advance. And they can learn it to a very limited degree, so what I said is accurate.

Encoding Grammar in Transformers: Gradient Descent Learning Challenges

By

–

21 November 2022 3h32

You can encode a grammar into a transformer. The problem is gradient descent is not very good at learning it.

Intelligence: Similarity and Compositionality in Transformer Models

By

–

21 November 2022 3h09

Intelligence is similarity + compositionality. Transformers' similarity is dot product of embeddings and their compositionality is attention, both still very primitive.

LLMs Face the Same Problem as NLP: Unclear Use Cases

By

–

20 November 2022 2h02

The problem with LLMs is what it's always been with NLP: if people can't tell what they're good for and not, they'll wind up not using them for much.

20 November 2022

LLMs Improving Faster Than Academic Publishing Quality

By

–

20 November 2022 2h00

Considering the speed at which LLMs are getting better and Nature/Science worse, it won't be long before they cross.