Sadly it doesn't support streaming tokens yet – I nearly got that working but ran into an API bug that I couldn't work around
TOOLS
-
New LLM Plugin llm-reka for Reka AI Models API
By
–
New LLM plugin: llm-reka – for running prompts against the @RekaAILabs family of models via their API https://
github.com/simonw/llm-reka pipx install llm
llm install llm-reka
llm keys set reka
# paste key here
llm -m reka-core '5 fun facts about walruses' -
API Streaming Support Request for AI Model Responses
By
–
Any plans to add streaming to your API? It looks like right now it only supports returning the entire response as a single chunk of JSON
-
Anthropic Needs Public Tokenizer API Like Google Gemini
By
–
Anthropic don't release any form of their tokenizer at all – not even an API endpoint that lets you count tokens before prompting with them (Google Gemini has that) It's a small thing but it's the feature I most want from them at the moment
-
Document Upload Transparency and AI Processing Visibility
By
–
I want to know exactly what happens to documents I upload to it – how they are split and chunked, how text is extracted from PDFs, what gets embedded and stored, and what gets retrieved and included in prompts (Ideally I'd like a debug interface that lets me see it for myself)
-
Anyscale Endpoints Plugin 0.5 Adds Mixtral 8x22B Support
By
–
New LLM plugin release: llm-anyscale-endpoints 0.5 adds support for their new hosted Mixtral 8x22B Instruct model, plus a "llm anyscale-endpoints refresh" command for refreshing the list of models without needing to upgrade the plugin in the future https://
github.com/simonw/llm-any
scale-endpoints/releases/tag/0.5
… -
AI Boosts Productivity: Happy Ramp Customer Shares Success
By
–
Love it when AI significantly improves productivity, and happy @tryramp customer here too!
-
H2O.ai Named Leader in Forrester Wave Computer Vision Tools Q1 2024
By
–
http://
H2O.ai is a LEADER in The Forrester Wave: Computer Vision Tools, Q1 2024 We received the highest scores possible in across a wide variety of criteria including data and model testing, innovation, and more! Follow the link to learn more: https://
h2o.ai/platform/Forre
ster/
… -
One Size Doesn’t Fit All in Technology Solutions
By
–
I don't know what my requirements are yet – I build a lot of different things with a lot of different people, so the only thing I'm truly confident about is that one size is not going to fit all of them
-
Claude 3 Haiku Image-to-Haiku Demos and Tools
By
–
Here's the full list of demos. My favourite is probably that first one – Haikus from images with Claude 3 Haiku – which you can try out yourself here if you have a Claude 3 API key https://
tools.simonwillison.net/haiku