Any plans to add streaming to your API? It looks like right now it only supports returning the entire response as a single chunk of JSON
AI
-
Quick Prompt to Assess Model Capability Level
By
–
Anyone got a good short go-to prompt for running against a model that instantly gives an indicator of how capable that model is? I'm trying to figure out a neat way of determining if a model feels gpt-3.5-turbo or gpt-4 level with the minimum amount of work (and API token spend)
-
RAG and Token Optimization with Expanded Context Windows
By
–
I was playing with maximizing RAG content last year, but that was when models still had tiny context lengths – 4,000 or 8,000 wasn't a lot to play with Token optimization like that is less interesting now we have 100,000+ tokens to play with even with the less expensive models
-
LLM Randomness: Breaking Out to NumPy for Random Choice
By
–
Whenever I want it to do the equivalent of random.choice, I break out to a tool that calls np.random. I'd love if I could just tell it to be random.
-

Exploring Tokenizer Vocabulary: Tool Call Token Structure
By
–
Here's how to start exploring the vocabulary of the tokenizer – interesting to see they have tokens like [TOOL_CALLS], [AVAILABLE_TOOLS], [/AVAILABLE_TOOLS], [TOOL_RESULTS] and [/TOOL_RESULTS] in there
-
Cost estimation and context optimization for advanced RAG queries
By
–
I want to be able to give users a cost estimate before they run larger prompts, but I'm also interested in knowing how much content I can cram into the context for advanced RAG queries
-
Anthropic Needs Public Tokenizer API Like Google Gemini
By
–
Anthropic don't release any form of their tokenizer at all – not even an API endpoint that lets you count tokens before prompting with them (Google Gemini has that) It's a small thing but it's the feature I most want from them at the moment
-
Mixtral Tokenizer Solves OpenAI’s JSON Token Counting Problem
By
–
OpenAI's tokenizer is available (as the tiktoken Python library) but you're on your own when it comes to guessing how many tokens will be used by the additional JSON syntax used for previous messages and tool definitions Mixtral's tokenizer here solves that problem
-

Databricks Marketplace Launches Premier Finance Datasets
By
–
Premier datasets in finance are now available on the #DatabricksMarketplace, thanks to @ICE_Markets
. These datasets enable key use cases like time-series analysis, relative value, spread analysis, and assessing climate risk exposure. http://
dbricks.co/4cYPFOx -
Waymo Driverless Vehicles Spotted on SF Highway 101
By
–
My neighborhood in SF, except for my block, is always thick with humanless Waymo vehicles. In tonight's rush hour I saw my first humanless Waymo on a highway, on 101, driving north to the city around the 280 interchange. I first took a ride in autonomous mode in a Waymo (then