MACHINE LEARNING

→ View original post on X — @perplexity_ai

Following Strategy for High Engagement Influencers in Tech

By

@perplexity_ai

–

20 December 2022 18h18

Or, were looking to follow new people who engage often with you – example, “bar plot top 20 @amasad doesn't follow but keeps them engaged” https://
perplexity.ai/sql?uuid=48f55
773-6276-4e22-9e13-287366e34159
…

→ View original post on X — @abhi1thakur

Hugging Face AutoTrain: Free Model Training Without Pro Subscription

By

@abhi1thakur

–

20 December 2022 17h24

Now you don't need a pro subscription on Hugging Face to train more than 5 models for a project using AutoTrain!!! However, if you have a PRO account, you can train up to 10 models per project for FREE! Try AutoTrain NOW: http://
hf.co/AutoTrain

→ View original post on X — @langchain

MMR Example Selector for Optimal Contextual Diversity

By

@langchain

–

20 December 2022 17h06

Max Marginal Relevance (MMR) Example Selector Based on this preprint by @xiye_nlp et al, the MMR example selector chooses diverse, in-context examples H/t @johnjnay for surfacing this paper Docs: https://langchain.readthedocs.io/en/latest/examples/prompts/prompt_management.html Original Paper: https://x.com/xiye_nlp/status/1603821850592628738?s=20&t=wQPcRd8IJ8URbmytuYP7QA

Gradient Backpropagation and Frequent Word Prioritization in Training

By

–

20 December 2022 16h13

Because we sum, the gradients for each token get backpropagated to all the rows that were used for it. So with enough training, the model ends up at a good compromise. Frequent words are naturally prioritised in this, because they'll simply have more gradients.

Hash Embedding Performance on Small Datasets Analysis

By

–

20 December 2022 16h13

However, the hash embedding doesn't do so well if the datasets are too small. That's one of the results that I found really useful in this tech report. It's great to have this all explored more thoroughly, and we look forward to publishing more experiments like this in future.

spaCy’s Hashing Approach for Word Representations Learning

By

–

20 December 2022 16h12

This is almost what we do in spaCy — it's just that, instead of reserving the first 9900 rows for common words, we just use the hashing. Given that the assignment of words to shared rows is arbitrary, how can the model learn good representations?

Hashing Unknown Words to Shared Vector Rows for Rare Words

By

–

20 December 2022 16h12

Imagine we gave the unknowns the last 100 rows instead. When we see an unknown word, we'll hash its word form, and mod the result to pick a row for it. Each unknown word will share its vector with lots of others, but its vector-mates are probably rare.

Hashing Trick for Fixed-Size Embedding Tables in Fine-Tuning

By

–

20 December 2022 16h12

Resizing the embedding table for fine-tuning is pretty awkward. We'd rather have some way of making sure that novel words can get a unique representation, even from a fixed-size embedding table. The hashing trick achieves this.

Embedding Table Strategy: Frequent Words and Shared Vectors

By

–

20 December 2022 16h12

Let's say we have 10k rows in the embedding table. In a standard embedding table, you give each of the 9999 most frequent words their own vector, and have all the others share the last vector.