Yes, def, did you see the snapdragon keynote? 30 tokens a second for Llama2 7B param on device. Maximum 13B param available to run on device. Apple can def run LLM’s on M2/M3
Snapdragon and Apple M2/M3 Running LLMs On-Device
By
–
By
–
Yes, def, did you see the snapdragon keynote? 30 tokens a second for Llama2 7B param on device. Maximum 13B param available to run on device. Apple can def run LLM’s on M2/M3