backend can be anything. im using vllm with llama 8b gguf locally

→ View original post on X — @abhi1thakur