Unfortunately, I have to agree with this. Especially w/ longer context. I would bet that this is either extremely quantized, or significantly smaller than older GPT-4s. It has moments of genius, for sure, and the speed is amazing. But it definitely loses some of its magic.
GPT-4 Performance Degradation: Quantization and Model Size Concerns
By
–