I haven't made a video but I've been writing a series of long-form posts about how I do AI-assisted development, most recent is this one:
@simonw
-
GPU Purchase vs Cloud Rental: Cost-Benefit Analysis for AI
By
–
Every now and then I have a quick poke around to see how much it would cost to buy a couple of NVIDIA cards and a PC to wire them together in, but the cost feels like I could get better value from renting out an A100 or similar for a few hours every now and then
-
ChatGPT Code Interpreter forgetting code execution ability
By
–
Anyone else noticed an uptick in ChatGPT with Code Interpreter forgetting it has the ability to run code? I've had quite a few sessions like this one recently https://
chat.openai.com/share/c1b6787d
-c25c-4baf-a083-fd83e9af4d8b
… -
MacBook Pro RAM Constraints for Running Mixtral 8x22B
By
–
Increasingly regretting only getting 64GB of RAM for my MacBook Pro M2 – it looks like Mixtral 8x22B needs 128GB
— Simon Willison (@simonw) 11 avril 2024
Hoping there will be quantized versions I can run soon https://t.co/PG9IqiBLewIncreasingly regretting only getting 64GB of RAM for my MacBook Pro M2 – it looks like Mixtral 8x22B needs 128GB Hoping there will be quantized versions I can run soon
-
Building AI Tools: Managing Model Mistakes and Reliability
By
–
Not at all. I'm building tools that work against multiple models, I'm happy to leave the detailed evals to other people. I'm going to tell my users that they should assume that AI makes mistakes all the time, because that's a pretty good rule of thumb for everything!
-
New llm-gemini Plugin Release with Embedding Model Support
By
–
Exactly what I needed, thank you! I just released a new llm-gemini plugin version with support for that embedding model: https://
github.com/simonw/llm-gem
ini/releases/tag/0.1a3
… -
Free API token limits and cost implications analysis
By
–
… but the 32,000 tokens per minute presumably prevents free API calls from using more than 32,000 tokens? If so max cost would be 22 cents per call for input, so a lot less expensive overall
-
API costs optimization: maximizing free tier usage limits
By
–
So I think the maximum a free call could cost you would be 1 million input / 8,192 output, which is $7 for the input and 17c for the output Times that by 50 and you could still burn $358/day if you were REALLY maxing out the possible prompt usage
-
Gemini Pro Output Token Limit Impact on Model Performance
By
–
Yeah the cap on output tokens is a really important variable in all of these – Gemini Pro's limit is 8,192
-
AI Data Extraction Accuracy Requires Human Review
By
–
I included the date mistake (assuming 2019 not 2024) in the video to try to help illustrate that – it's generally great at extracting things correctly if everything is present in the text, but you still have to review what it did because there are many ways things could go wrong