Totally, depends on the usecase and the number of users you concurrently want to serve. I get 20-25 tokens/sec on my Macbook M1 pro 16 GB.
MacBook M1 Pro achieves 20-25 tokens per second performance
By
–
By
–
Totally, depends on the usecase and the number of users you concurrently want to serve. I get 20-25 tokens/sec on my Macbook M1 pro 16 GB.