OPEN SOURCE

→ View original post on X — @mattshumer_

Llama 3 Open Source Release Announced Within 24 Hours

By

@mattshumer_

–

18 April 2024 5h26

less than 24 hours till llama 3 is open-sourced

Mixtral 8x22B Now Available via Mistral AI Platform API

By

–

18 April 2024 5h15

Mixtral 8x22B is now available via the @MistralAILabs La Platforme API If you previously installed the llm-mistral plugin run "llm mistral refresh" to refresh the list of available models – otherwise a fresh install will provide it Released 0.3.1 anyway: https://
github.com/simonw/llm-mis
tral/releases/tag/0.3.1
…

Open Source Models Prompt Injection Security Flaw

By

–

18 April 2024 4h46

Yeah, I wouldn't trust that! That does highlight interesting flaw in a lot of open models though: I think there are some models that use strings like [INST] without even reserving a token for them, which opens up all sorts of additional potential prompt injection mischief

New LLM Plugin llm-reka for Reka AI Models API

By

–

18 April 2024 4h33

New LLM plugin: llm-reka – for running prompts against the @RekaAILabs family of models via their API https://
github.com/simonw/llm-reka pipx install llm
llm install llm-reka
llm keys set reka
# paste key here
llm -m reka-core '5 fun facts about walruses'

Mixtral Tokenizer Solves OpenAI’s JSON Token Counting Problem

By

–

18 April 2024 2h31

OpenAI's tokenizer is available (as the tiktoken Python library) but you're on your own when it comes to guessing how many tokens will be used by the additional JSON syntax used for previous messages and tool definitions Mixtral's tokenizer here solves that problem

Anyscale Endpoints Plugin 0.5 Adds Mixtral 8x22B Support

By

–

18 April 2024 1h00

New LLM plugin release: llm-anyscale-endpoints 0.5 adds support for their new hosted Mixtral 8x22B Instruct model, plus a "llm anyscale-endpoints refresh" command for refreshing the list of models without needing to upgrade the plugin in the future https://
github.com/simonw/llm-any
scale-endpoints/releases/tag/0.5
…

→ View original post on X — @jxmnop

PyTorch DDP Gradient Syncing Bug in Distributed Training

By

@jxmnop

–

18 April 2024 0h13

so last week I was posting about a bug i had with gradient syncing in torch DistributedDataParallel (DDP), aka the most standard way to do multi-GPU training for neural networks all of my misery originated with a single design decision from the torch team: DDP shares