To train this model, we use Curriculum Learning by gradually increasing the token lengths trained on from 2K to 8K. We additionally train on an instruction tune dataset sampled from various popular sources and curated in-house. (4/10)
AI
-
Dataset Curation for Long Sequence Instruction Following
By
–
To curate a dataset that encourages long sequence instruction following, we use two techniques. First we find tasks that truly benefit from longer sequences and add them to our instruction tuning datasets. (5/10)
-
Instruction List Technique Enhances Long Sequence Attention
By
–
Second, we use a technique that we developed called instruction list that synthetically creates tasks that encourage more long sequence attention and instruction following. (6/10)
-
13B Parameter Long Sequence Model Achieves Competitive Accuracy
By
–
Using this recipe, we are able to train a competitive long sequence model at 13B parameter scale. We achieve 2-12 points better accuracy across a wide variety of long sequence tasks from Scrolls and ZeroScrolls. (7/10)
-
SN-13B-8K-Instruct Outperforms MPT, XGen, LLAMA2 on Long Sequences
By
–
We compare SN-13B-8K-Instruct with MPT, XGen and LLAMA2 and find that this model achieves better scores on long sequence suite derived from Scrolls and validation set of ZeroScrolls, a benchmark developed by @TelAvivUni and @MetaAI
. (3/10) -
SN-13B-8K-Instruct Open Source Model Outperforms Long Sequence Competitors
By
–
Today we are excited to introduce and open source SN-13B-8K-Instruct, an 8K SS model that achieves better accuracy than leading long sequence open source models specialized for long sequence size tasks. (1/10)
-
Sambanova SN-13B-8k-Instruct Model Now Available for Download
By
–
This model is available for download on @huggingface for people to try: https://
huggingface.co/sambanovasyste
ms/SN-13B-8k-Instruct
… (2/10) -
Zoom’s Data Usage Policies Raise Privacy and AI Ethics Concerns
By
–
That's cool of Bard, and scary from Zoom. "We can use your meetings for…pretty much anything we want to"
-
Installation and requirements setup for deep learning workshop
By
–
Maybe this installation + requirements.txt from my recent DL workshop is useful here: https://
github.com/rasbt/scipy202
3-deeplearning/tree/main/00-2_python-libraries-for-workshop
… -
Working at Ramp: Company Culture and Employee Experience
By
–
What it's like to work at @tryramp pic.twitter.com/QXyT2j3xaG
— Lenny Rachitsky (@lennysan) 7 août 2023What it's like to work at @tryramp