LangChain v0.0.42 Doc fixes by @YouSearchEngine team, @_abi_ Jinja formatting – @benderville Max Marginal Relevance example selector (based on paper by @xiye_nlp Caching improvements (h/t @devonbrackbill for idea) These are sneaky some awesome updates, lets
CODE
-
spaCy’s Hashing Approach for Word Representations Learning
By
–
This is almost what we do in spaCy — it's just that, instead of reserving the first 9900 rows for common words, we just use the hashing. Given that the assignment of words to shared rows is arbitrary, how can the model learn good representations?
-
Hashing Unknown Words to Shared Vector Rows for Rare Words
By
–
Imagine we gave the unknowns the last 100 rows instead. When we see an unknown word, we'll hash its word form, and mod the result to pick a row for it. Each unknown word will share its vector with lots of others, but its vector-mates are probably rare.
-
Hashing Trick for Fixed-Size Embedding Tables in Fine-Tuning
By
–
Resizing the embedding table for fine-tuning is pretty awkward. We'd rather have some way of making sure that novel words can get a unique representation, even from a fixed-size embedding table. The hashing trick achieves this.
-
Embedding Table Strategy: Frequent Words and Shared Vectors
By
–
Let's say we have 10k rows in the embedding table. In a standard embedding table, you give each of the 9999 most frequent words their own vector, and have all the others share the last vector.
-
Improving Vector Representation for Unknown Words in NLP
By
–
This is kind of weird, if you think about it. The unknown words together will be more frequent than the 9998th most frequent term. So the fidelity of representation isn't being distributed well. How can we give the unknowns more vectors?
-
Word representations learning for smaller datasets and per-token decisions
By
–
For language modelling and translation, word pieces are the standard way to address this. But for smaller datasets, especially where the decisions need to be made on a per-token basis, it's very helpful to learn representations for words, rather than characters or word pieces.
-
spaCy Trained Pipelines: Domain Flexibility and Fine-tuning Capabilities
By
–
But for spaCy, we want to give people trained pipelines and have them be as applicable as possible across different domains. We also want people to be able to fine-tune the models on their data if it's not working ideally.
-
The Hashing Trick in spaCy Models: An Underexplored Technique
By
–
The hashing trick is an old technique from sparse linear models, well known in toolkits like Vowpal Wabbit. It's one of the unusual things I did in @spacy_io 's models that I've always felt was quite neat, and needed more experiments and write up.
-
Unbounded Vocabularies and Fixed-Size Embedding Tables Explained
By
–
The basic motivation is that vocabularies are unbounded, but embedding tables can only be a fixed size. If you know your training data matches up to your test data well, this isn't such a big deal for most applications. If it's rare at training time, it'll be rare at test time.