From the LMSYS site: "Model providers can test their unreleased models anonymously, meaning the models' names will be anonymized." I interpret that as covering both battle mode and direct chat mode
@simonw
-
LMSYS Anonymized Model Names and OpenAI Attribution
By
–
The LMSYS explanation is that they pick "anonymized" names for new blind tested preview models, so it's possible OpenAI themselves didn't pick this name (assuming it's one of theirs, which still seems likely to me)
-
Tokenizer Design Evolution: Why Revisit Outdated 2019 Architecture?
By
–
Why would they do that though? The state of the art in tokenizers has moved on enormously since 2019, why go back to an old design?
-
Prompt Optimization and RAG Over Model Fine-Tuning
By
–
I'm not patient enough to fine-tune my own models – plus I have a hunch that for most applications you can get better results from prompt optimization and techniques like RAG that don't require any additional fine-tuning at all
-
Improving AI Policy Transparency for Model Users
By
–
I like that it's at least documented as a policy on https://
lmsys.org/blog/2024-03-0
1-policy/#our-policy
… – a small improvement could be provide a link to that policy that shows up when you're interacting with a model covered by it, to make things less confusing and opaque -
Plugin Model Triggers Code Interpreter via Assistants API
By
–
Huh… a plugin that provides a model that triggers Code Interpeter via the assistants API could be neat…
-
Using Claude, GPT-4, Haiku, Llama and Phi for AI workflows
By
–
Opus via LLM for CLI stuff, GPT-4 via ChatGPT for Code Interpreter and general convenience Haiku as an API for some features I'm building, Llama 3.8B and Phi-3 for running in my laptop I'm starting to try http://
meta.ai and Google Gemini for search-backed queries -
LMSYS Blind Test Mode Policy Documentation and Guidelines
By
–
I hadn't seen this before: LMSYS have clear documentation of their policies around this kind of "blind test mode" on their site: https://
lmsys.org/blog/2024-03-0
1-policy/#our-policy
… -
AI System Improves Accuracy in Personal Information Handling
By
–
Huh, interesting – in the past it's usually put in incorrect details like saying I was CTO at GitHub or Eventbrite, maybe it's got better since the last update
-
LMSYS Production Configuration Transparency Open Source
By
–
Yeah possibly? I don't know much about how lmsys are configured – wish their exact production configuration was open source!