The fallback function of Siri, Alexa etc is to search Google or Wikipedia. When that happens, it's pretty obvious. But with ChatGPT, it's extremely murky (to me at least) — so much so that I don't even know how important the explicit supervision really is. 4
@honnibal
-
Voice Assistants Limited by Predefined Commands Not Natural Language
By
–
Siri, Alexa etc have a bunch of specific functionality, and language is just the UI. It's like typing into one of those old adventure games. If the game wants you to catch the cat with the flypaper or whatever, you can't crouch down and slow blink to lure it over instead. 3
-
Distinguishing Emergent vs Supervised Behaviors in ChatGPT
By
–
Some of the things ChatGPT can do are emergent behaviours, picked up from the language or code modelling. Other things it can do have been specifically taught to it. Is there some speculative list somewhere about what tasks were supervised? 1
-
Understanding Emergent Capabilities in AI Technology
By
–
If we knew which capabilities are fully emergent, that would help a lot in guessing the trajectory of the underlying technology, and I'd find it a bit easier to use.
It's like when Siri came out and lots of people didn't get how it worked at all. 2 -
Transformers and LSTMs: Scaling architectures and the need for innovation
By
–
I agree with the core thing I think you’re saying: there’s lots of architectures that could allow models to scale with respect to both data and computation. Transformers are one, maybe not even a good one. But LSTMs as we have then don’t scale — so we were missing an invention
-
Word embeddings and latent factors in language models
By
–
I'd argue that the janky "queen = king – man + woman" thing is definitely related. It's not doing language generation, but you're trying to query the latent factors the model has used to minimise the surface objective
-
Using nlp.pipe() for faster batched predictions in spaCy
By
–
Nice example! Here's a tip for next time as well: the nlp.pipe() function operates over a sequence of inputs, and lets the model make batched predictions, making it much faster:
-
Video remains most complete explanation, written version pending
By
–
Unfortunately my video on it remains the most complete explanation of it: https://
youtube.com/watch?v=sqDHBH
9IjRU
…
I remain apologetic that I haven't gotten this out in a written form! It's a lot to explain fully, and at all times other things have come up. I'm happy to answer questions -
Zshot: Linking Entity Mentions to Exemplars via Transformers
By
–
Zshot uses a transformers model that links mentions to exemplars you provide to the pipeline component. You can use standard NER mentions with @spacy_io 's built-in models, or you can train your own with https://
prodi.gy, optionally assisted by @OpenAI suggestions. -
spaCy Extension for Zero-Shot Labeling with Prodigy Integration
By
–
Following up with the theme of zero and few-shot labelling, this spaCy extension by @IBMResearch is super cool, and could be used in combination with our recent Prodigy recipe: https://
github.com/explosion/prod
igy-openai-recipes
…