Not saying not to use it, but expect inaccuracies Doing this right requires actual hardware + knowing which piece of the inference stack to optimize for the different hardware & goal We are still in the infancy stage of opensource inference, one day it will just work OOTB

→ View original post on X — @theahmadosman