Here's my current documentation on using LLM with these kinds of endpoints https://
llm.datasette.io/en/stable/othe
r-models.html#openai-compatible-models
…
@simonw
-
LLM Documentation OpenAI Compatible Endpoints Guide
By
–
-
Standardizing Widely-Used Patterns: Call for Specification
By
–
This pattern is so widely used at this point that it would be nice if someone could spin it off into a separate, thoroughly documented specification for people to follow – maybe with a new name? Would be good to have the same thing for S3 as well
-
Adding OpenAI-compatible endpoints to Datasette LLM tool
By
–
Context is I want to add more features relating to this to https://
llm.datasette.io so I'd like to use language that's widely understood but also as concise as possible llm openai-compatible-endpoints add perplexity https://… is a bit much! -
Naming OpenAI Compatible LLM API Endpoints Pattern
By
–
What's a good name for that pattern where LLM API providers have endpoints that speak the same JSON protocol as OpenAI, and can be called using the OpenAI client libraries? I've been calling them "OpenAI compatible endpoints" but that's a bit of a mouthful
-
OpenAI API Features Discovery and Developer Learning
By
–
I had genuinely no idea that was a thing for the OpenAI APIs
-
Prompt Injection Security: Why Reducing Attack Success Rate Isn’t Enough
By
–
That's the key challenge with prompt injection: reducing to a tiny probability isn't good enough because this is a security vulnerability: if only 1/1000 attacks work then an adversarial attacker will find still find the ones that do
-
Whisper Training Data Source YouTube Analysis
By
–
I've seen people say that's because Whisper was trained on huge amounts of YouTube, which feels feasibly to me
-
ChatGPT Voice Hallucinations from Background Noise Fragments
By
–
It's quite fun leaving ChatGPT Voice running and later seeing how it hallucinated increasingly weird fragments of conversation based on snippets of background noise
-
Voice assistants home usefulness concept failure analysis
By
–
Do you think that was a failing in the technology or a failing in the concept itself? Is it possible that a voice assistant in your home is actually much less useful and compelling than it seems like it should be?
-
Llama 3 token vocabulary expansion optimization benefits
By
–
I don't think I've talked about going beyond tokenization? Still seems like a sensible approach to me – Llama 3 upped their token vocabulary from 32,000 to 128,000 which gave them some optimization benefits