OK, can confirm that it works on an M2 with 64GB of RAM I had to quit Firefox and VS Code to free up enough RAM for it to run – Meta-Llama-3-70B-Instruct.Q4_0.llamafile now uses 38GB of RAM and runs at about 7.5 tokens a second
COMPUTING
-
Edge AI Adoption: Real-Time Decision-Making Through Model Optimization
By
–
The Promise of Edge AI and Approaches for Effective Adoption: Organizations are adopting edge AI for real-time decision-making using efficient and cost-effective methods such as model quantization, multimodal databases, and distributed inferencing. https://
kdnuggets.com/the-promise-of
-edge-ai-and-approaches-for-effective-adoption?utm_source=dlvr.it&utm_medium=twitter&utm_campaign=the-promise-of-edge-ai-and-approaches-for-effective-adoption
… -
Default Settings Change: Yes to No Paradigm Shift
By
–
I think changing default to no instead of yes is the biggest change one can do.
-
Machine Learning YouTube Courses Repository for Data Science
By
–
A Repository of #MachineLearning YouTube Courses! #BigData #Analytics #DataScience #AI #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #CloudComputing #Serverless #DataScientist #Linux #Programming #Coding #100DaysofCode https://
geni.us/YouTube-Courses -
Essential AWS Books for Data Science and Cloud Computing
By
–
The Top Amazon #WebServices #Books You Need to Read! #BigData #Analytics #DataScience #IoT #IIoT #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #CloudComputing #Serverless #DataScientist #Linux #Books #Programming #Coding #100DaysofCode https://
geni.us/AWS-Need -
HTML Cheat Sheet: Essential Guide for Web Development
By
–
Cheat-Sheet! #HTML! #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #CloudComputing #Serverless #DataScientist #Linux #Programming #Coding #100DaysofCode https://
geni.us/HTML-Cheat-Sh -
ARPA and Military Origins of Networked Computing Explained
By
–
(Yes, gentlemen, I know about ARPA and the US-military origins of networked computation. But I'm sure there's someone else out there who'd love an explanation.)
-
AI as Key to Understanding Human Brain Function
By
–
AI might be the only way humans can understand their own brains better with.
-
CPU GPU Monitoring Tool Software Identification Question
By
–
What program are you using to produce that CPU/GPU utilization info on the left?
-
M2 64GB MacBook Compatibility for AI Workloads
By
–
Your docs say "Don't bother if you're using a Macbook M1 with 32GB of RAM" – any idea if an M2 64GB can handle it?