#opensource wins, thanks @sallywf @eetimes
OPEN SOURCE
-
Dolma: A 3 Trillion Token Open-Source Training Dataset for LLMs
By
–
A 3 Trillion (yes, trillion) token open-source training dataset for LLMs: Dolma is built from a mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials. Now available on Huggingface if you have a spare 5.4 Terabytes
-
Allen AI releases OLMo open language model with technical documentation
By
–
Technical Report: https://
allenai.org/olmo/olmo-pape
r.pdf
…
Blog: https://
blog.allenai.org/olmo-open-lang
uage-model-87ccfc95f580
…
Github: https://
github.com/allenai/OLMo
Models: -
AI2 Launches OLMo Open Language Model Framework
By
–
@allen_ai launched "OLMo: Open Language Model". AI2 opens its framework that provide access to data, training code, models, and evaluation code. The training data Dolma is also released, a dataset of 3 trillion tokens from a diverse mix of web content, academic
-
Meta Explains AI Open-Source Strategy in Earnings Call
By
–
If you have questions about why Meta open-sources its AI, here's a clear answer in Meta's earnings call today from @finkd
-
Ludwig Celebrates 10k Milestone with LLM Hackathon Winners
By
–
Congrats to our #LLM Hackathon Winners! To celebrate Ludwig's 10k we ran a mini virtual hackathon. Projects include: Automating data quality checks in Python #QA for Indian Tax Code #Classification for Customer Service
and more! https://
pbase.ai/3HGSQMo -
Allen AI Releases Training Data Scripts and Code Openly
By
–
very cool to release training data scripts and training code!!!@allen_ai is pushing the limits of openness, great work. https://t.co/vY9thKyVGd
— Soumith Chintala (@soumithchintala) 1 février 2024very cool to release training data scripts and training code!!! @allen_ai is pushing the limits of openness, great work.
-
Building Long-Context RAG with Nomic Embeddings and Mistral
By
–
Build Long-context RAG from scratch: Nomic Embeddings + Mistral The context window of open source LLMs and embedding models has been relatively small vs proprietary models. But, methods to expand context window (RoPE, self-extend) are quickly changing this. Today, @nomic_ai has
-
Fine-tuning Mistral 7B with DPO dataset for improved chat performance
By
–
I fine-tuned Mistral7Bv0.2 locally with MLX using this DPO dataset by @argilla_io
, composed of 7k chat interactions distilled from Capybara. I haven't done a full evaluation yet, but for this size, it's really good, and it understood this famous question right away! Link -
Nomic Launches langchain-nomic Python Package for Embeddings
By
–
@nomic_ai Embeddings Today we're thrilled to unveil the new langchain-nomic Python package, which integrtes with Nomic's Embedding API. Experience the power of embedding models that deliver results competitive with OpenAI's new text-embedding-3 models.