less than 24 hours till llama 3 is open-sourced
LLMS
-
Mixtral 8x22B Now Available via Mistral AI Platform API
By
–
Mixtral 8x22B is now available via the @MistralAILabs La Platforme API If you previously installed the llm-mistral plugin run "llm mistral refresh" to refresh the list of available models – otherwise a fresh install will provide it Released 0.3.1 anyway: https://
github.com/simonw/llm-mis
tral/releases/tag/0.3.1
… -
Open Source Models Prompt Injection Security Flaw
By
–
Yeah, I wouldn't trust that! That does highlight interesting flaw in a lot of open models though: I think there are some models that use strings like [INST] without even reserving a token for them, which opens up all sorts of additional potential prompt injection mischief
-
New LLM Plugin llm-reka for Reka AI Models API
By
–
New LLM plugin: llm-reka – for running prompts against the @RekaAILabs family of models via their API https://
github.com/simonw/llm-reka pipx install llm
llm install llm-reka
llm keys set reka
# paste key here
llm -m reka-core '5 fun facts about walruses' -
Quick Prompt to Assess Model Capability Level
By
–
Anyone got a good short go-to prompt for running against a model that instantly gives an indicator of how capable that model is? I'm trying to figure out a neat way of determining if a model feels gpt-3.5-turbo or gpt-4 level with the minimum amount of work (and API token spend)
-
RAG and Token Optimization with Expanded Context Windows
By
–
I was playing with maximizing RAG content last year, but that was when models still had tiny context lengths – 4,000 or 8,000 wasn't a lot to play with Token optimization like that is less interesting now we have 100,000+ tokens to play with even with the less expensive models
-
LLM Randomness: Breaking Out to NumPy for Random Choice
By
–
Whenever I want it to do the equivalent of random.choice, I break out to a tool that calls np.random. I'd love if I could just tell it to be random.
-
Exploring Tokenizer Vocabulary: Tool Call Token Structure
By
–
Here's how to start exploring the vocabulary of the tokenizer – interesting to see they have tokens like [TOOL_CALLS], [AVAILABLE_TOOLS], [/AVAILABLE_TOOLS], [TOOL_RESULTS] and [/TOOL_RESULTS] in there
-
Cost estimation and context optimization for advanced RAG queries
By
–
I want to be able to give users a cost estimate before they run larger prompts, but I'm also interested in knowing how much content I can cram into the context for advanced RAG queries
-
Anthropic Needs Public Tokenizer API Like Google Gemini
By
–
Anthropic don't release any form of their tokenizer at all – not even an API endpoint that lets you count tokens before prompting with them (Google Gemini has that) It's a small thing but it's the feature I most want from them at the moment