Which normal person understands quantization? I work in tech and I barely know what it means let alone my mom?
@levelsio
-
Open Models: UX and Value Creation Strategy
By
–
Models are free and open, you're responsible for building a great user experience around that That's the value
-
LM Studio accessibility issues for non-technical users
By
–
LM Studio is great but unusable for non-techies, it's still very hard You have to download it, install it, go to the right tab, search the model, choose the right quantization (this is insane btw, why doesn't it choose the best for me) It's very engineer minded and not user
-
Native iOS Apps and Client-Side LLM Implementation
By
–
A native iOS app that does client-side LLM would have to do same so no diff here
-
Cloud to Local AI Model Switching Strategy
By
–
One way to deal with the limitations of having to download model – let user use cloud model but show an indicator Cloud Based Conversation or smth then once it's locally ran switch to 100% private local conversation
– if user GPU or connection doesn't allow for locally run -
Smaller Models and Bigger GPUs Reshape AI Hardware Future
By
–
Yep, but models will get smaller and GPUs bigger esp now that AI is so popular I think
-
GPU and Internet Requirements for Local AI Model Deployment
By
–
That should NEVER even end up at tech suport You should check if user has 1) GPU to run it fast 2) internet speed to download model fast If not you should not even let them use it and just give them the cloud version instead
-
Client-Side AI Execution: GPU and Model Size Constraints
By
–
Essentially there's nothing you can NOT run client-side now The only limit is GPU size and model size to download