Second, we use a technique that we developed called instruction list that synthetically creates tasks that encourage more long sequence attention and instruction following. (6/10)
@sambanovaai
-
Dataset Curation for Long Sequence Instruction Following
By
–
To curate a dataset that encourages long sequence instruction following, we use two techniques. First we find tasks that truly benefit from longer sequences and add them to our instruction tuning datasets. (5/10)
-
Curriculum Learning Boosts Model Training From 2K to 8K Tokens
By
–
To train this model, we use Curriculum Learning by gradually increasing the token lengths trained on from 2K to 8K. We additionally train on an instruction tune dataset sampled from various popular sources and curated in-house. (4/10)
-
SN-13B-8K-Instruct Outperforms MPT, XGen, LLAMA2 on Long Sequences
By
–
We compare SN-13B-8K-Instruct with MPT, XGen and LLAMA2 and find that this model achieves better scores on long sequence suite derived from Scrolls and validation set of ZeroScrolls, a benchmark developed by @TelAvivUni and @MetaAI
. (3/10) -
Sambanova SN-13B-8k-Instruct Model Now Available for Download
By
–
This model is available for download on @huggingface for people to try: https://
huggingface.co/sambanovasyste
ms/SN-13B-8k-Instruct
… (2/10) -
SN-13B-8K-Instruct Open Source Model Outperforms Long Sequence Competitors
By
–
Today we are excited to introduce and open source SN-13B-8K-Instruct, an 8K SS model that achieves better accuracy than leading long sequence open source models specialized for long sequence size tasks. (1/10)
-
Meta Open-Sources AI Model for Developer Innovation
By
–
“This transition opens possibilities for democratizing AI and enabling enterprises to build custom software on top of the technology. By open-sourcing this model, as well as making it free, Meta allows researchers and developers to build on, deconstruct and learn from its
-
SambaNovaAI DataScale SN30 Training at Argonne Testbed
By
–
Join us for a 2-day training that will introduce users to the @SambaNovaAI DataScale SN30 system architecture and software stack, SambaFlow. The session will provide hands-on training to get started on the SN30 in the @argonne_lcf AI testbed. The event will be virtual and the
-
SambaNovaAI DataScale SN30 Training Workshop at Argonne
By
–
Register now for an exciting training workshop introducing users to the @SambaNovaAI DataScale SN30 deployed at the @argonne_lcf AI Testbed. This workshop is targeted at universities, national labs, and industries working on open science. https://
alcf.anl.gov/events/sambano
va-ai-training-workshop-0
… Spread the -
SambaNova AI Multi-Rack Deployment Powers Accenture Generative Solutions
By
–
Adding some fresh color with a new multi-rack install at our partner @Accenture. Fully integrated generative AI services and solutions provided by @Accenture, powered by @SambaNovaAI. #gpt #nlp #GenAI pic.twitter.com/tFD81QdCOH
— SambaNova (@SambaNovaAI) 12 juin 2023Adding some fresh color with a new multi-rack install at our partner @Accenture
. Fully integrated generative AI services and solutions provided by @Accenture
, powered by @SambaNovaAI
. #gpt #nlp #GenAI