Unfortunately it was a oversight since historically only ~0.5% of prompts coming into our system were super long and we just weren't thinking about it much
System Oversight: Long Prompts Underestimated in AI Infrastructure
By
–
By
–
Unfortunately it was a oversight since historically only ~0.5% of prompts coming into our system were super long and we just weren't thinking about it much