LLMS

→ View original post on X — @guillaumelample

Mistral Releases New 3B and 8B Parameter Models

By

@guillaumelample

–

16 October 2024 17h00

We just released two small models, with 3B and 8B parameters. Ministral 3B is exceptionally strong, outperforming Llama 3 8B and our previous Mistral 7B on instruction following benchmarks. https://
mistral.ai/news/ministrau
x/
…

DAM Optimization Outperforms Techniques for SQL and German Tasks

By

–

16 October 2024 15h49

Case study 2: SQL + German + Korean We combined three Llama-based models and optimized coefficients on three different datasets. DAM outperforms other techniques on average: in particular in German and SQL, but falls short in Korean.

→ View original post on X — @abhi1thakur

Hugging Face Hub Documentation for Ollama

By

@abhi1thakur

–

16 October 2024 15h49

doc: https://huggingface.co/docs/hub/en/ollama

→ View original post on X — @abhi1thakur

Run Any GGUF Model from Hugging Face Hub with Ollama

By

@abhi1thakur

–

16 October 2024 15h47

Now you can run any GGUF model from Hugging Face Hub with Ollama

DAM Model Merging: Japanese Math LLM Optimization Results

By

–

16 October 2024 15h43

Case study 1: Japanese + Math model We combined shisa-gamma-7b, WizardMath-7B-V1 and Abel-7B-002 and optimized coefficients on Ichikara, MetaMathQA, and Orca-Math. DAM achieved the best average score across various tasks, matching Evolutionary Merging but with less

DAM: Scaling Coefficients for Source Model Weight Merging

By

–

16 October 2024 15h42

DAM learns scaling coefficients for each column in the weight matrices of the source models. The objective function is optimized with gradient descent. It includes: – KL divergence to align the merged model's predictions with each expert source model
– Cosine similarity

Differentiable Adaptive Merging: Faster Automated Model Fusion

By

–

16 October 2024 15h42

Differentiable Adaptive Merging I participated in a paper with @arcee_ai about a new automated merging technique (no manual parameter tuning!) Compared to other algorithms, like Evolutionary Merging, it is significantly faster to run and achieves the same level of performance.

→ View original post on X — @h2oai

H2O AI Partners with AI Verify Foundation for Responsible AI Testing

By

@h2oai

–

16 October 2024 5h56

We’re partnering with the AI Verify Foundation to advance responsible AI adoption in Singapore! We’re contributing benchmarks and code to AI Verify’s open-source Project #Moonshot toolkit for LLM application testing – https://
h2o.ai/blog/2024/h2o-
ai-collaboration-with-ai-verify-foundation/
…