Get a fourth RTX PRO 6000 so you can run TP=4, you’ll have a lot more options and flexibility that way
@theahmadosman
-

Memo: Open-source AI will win
By
–
MEMO INCOMING re: Sovereign Intelligence Opensource AI WILL WIN x.com/TheAhmadOsman/…
-

Opensource AI Will Win: A Memo on Sovereign Intelligence
By
–

MEMO INCOMING re: Sovereign Intelligence Opensource AI WILL WIN
-

Free online book covers LLM concepts for all backgrounds
By
–
UNBELIEVABLE RESOURCE The bible for understanding LLMs is NOW AVAILABLE online to read (FOR FREE) Covers all the concepts below, no experience needed and anyone from any background can understand it – Tokens / Tokenizers
– Transformers
– Attention
– KV Cache
– Prefill vs -
Run local air-gapped models to avoid data center risks
By
–
Bruh, just run the model locally in an air-gapped server access w/ those Chinese models and you'll be fine since there is no internet connection Sending your life to a data center that someone controls IS NOT okay
-
Apology and warning: learn local LLMs to avoid rug pulls
By
–
I am sorry man, I keep telling people they must learn how to run LLMs locally and be ready to get rugpulled at any moment by these companies
-
Opensource AI needed to prevent financial leaks and dystopian future
By
–
Leaks and court cases showing people's finances will be a normal thing in this dystopian future of theirs That's why Opensource AI MUST WIN
-

Free online LLM bible: Understand ChatGPT, run models at home, future AI careers
By
–
DROP EVERYTHING The bible for how LLMs work is now available online to read FOR FREE This is for you if you: – Want to run these LLMs at home on your hardware?
– Want to understand how ChatGPT works?
– Want to work at an AI Lab in the future? Covers all the concepts from -
Inquiry about safetensors version availability vs ggufs
By
–
Do you have a safetensors version published or only ggufs?
-
Thousands of tokens per second across parallel requests
By
–
1000s of toks/sec across a dozen parallel requests if not more