What looks like ontology may be eigenspectrum. A beautiful new paper by Andres Nava and Matthieu Wyart gives a mechanistic account of one of the most striking facts about language models: semantic hierarchies appear geometrically. An owl is a bird.
A bird is an animal.
An
Paper: Semantic Hierarchies Are Geometric in Language Models
By
–
