On Windows you would need dedicated VRAM on a GPU I think, which is a whole lot more expensive – not sure how many NVIDIA cards you can fit in a laptop these days
@simonw
-
Running Model on 128GB Apple Combined Memory Setup
By
–
I saw someone run it on 128GB of Apple combined memory earlier https://t.co/PG9IqiBLew
— Simon Willison (@simonw) 11 avril 2024I saw someone run it on 128GB of Apple combined memory earlier
-
Undocumented AI Capabilities: System Prompt Knowledge Mystery
By
–
Yeah I've been wondering that for a while – nothing in the system prompt tells it how to use the /mnt folder to return links to files to download, but it clearly knows how to do that
-
Company Proxying AI Model Inference Through Fireworks and Together
By
–
Looks to me like they're proxying to Fireworks and Together rather than hosting themselves
-
Fireworks AI Launches Competitive Pricing at $0.90/Million Tokens
By
–
Fireworks AI for $0.90/million tokens
-
DeepInfra Offers Competitive Pricing at $0.65 per Million Tokens
By
–
@DeepInfra for $0.65/million tokens
-
Perplexity API availability status and playground UI features
By
–
Doesn't look like Perplexity offer it through their API yet, just through their playground UI – unless it's available via API but they haven't updated this page yet
-
Model not found on Fireworks AI platform list
By
–
I can't find it on the list on https://
fireworks.ai/models -
Together.ai offers LLM API at $1.20 per million tokens
By
–
Looks like https://
together.ai have it for $1.20/million tokens -
Which LLM API Providers Offer the New Mixtral Model?
By
–
Which of the LLM API providers have the new Mixtral model?