Could be wrong, but I think AI5 will probably be the best inference chip of any kind for models below ~250B params. By far lowest cost silicon and best performance/Watt. AI6 will take that much further.
AI5 and AI6 Inference Chips: Performance and Cost Leadership
By
–