4o is pretty great with individual queries, first turn-type stuff, etc. I'm guessing the majority of lmsys votes are on shorter conversations. It falls apart when longer context is needed.
@mattshumer_
-
Small AI Model Impressive But Not Better
By
–
To be clear, ESPECIALLY if it’s super small, the model is insanely impressive. But it is not *better*.
-
Meta’s Omni Models Challenge OpenAI’s GPT-4o
By
–
Meta has had GPT-4o-style omni models for at least five months. I’d be shocked if they don’t release an answer to GPT-4o soon. Imagine what a world with open-source omni models would look like…
-
GPT-4 Performance Degradation: Quantization and Model Size Concerns
By
–
Unfortunately, I have to agree with this. Especially w/ longer context. I would bet that this is either extremely quantized, or significantly smaller than older GPT-4s. It has moments of genius, for sure, and the speed is amazing. But it definitely loses some of its magic.
-
LMSYS Could Monetize Preference Data Through Reward Model Licensing
By
–
lmsys is sitting on a gold mine. If they train on all of the preference data they’ve collected, they will have (by far) the best AI-agnostic reward model in the world. License that out to model developers — they’ll pay a ton, as more robust preference models = far better AIs.
-
Nous Research merges Hermes 2 Pro and LLaMA 3 with better performance
By
–
Holy shit @NousResearch merged Hermes 2 Pro + LLaMA 3, and it performs even better than either model on their own
-
New Personas Feature Enhances AI Examples and Results
By
–
Haha sounds good. This can be way better w/ a few examples plugged into our new Personas feature!
-
Newsletter Ads ROI for Consumer Startups: Cost-Effective Growth?
By
–
Serious q: how do newsletter ads make sense for consumer startups? One reached out and said a HyperWrite competitor got ~400 users at ~$10/user. At that cost, solely growing via newsletters would cost tens of millions to reach our size. Please explain how this makes sense.
-
AI Model Image Capabilities Not Yet Released
By
–
This definitely isn’t the same model. The image capabilities haven’t been released.
-
Best AI Models Strategy Free Tier Deployment Planning
By
–
Also — no reason to give your best model to the free tier if you don’t plan to have a better model very soon