You’re misreading me — I’m saying ChatGPT data isn’t plausibly the source of DeepSeek-V3’s performance. The base model itself is good (as seen in its Pile-test scores, which match Llama 3.1 405B) and you can’t achieve that by training on ChatGPT.
LLMS
-
System prompts influence model responses
By
–
It’s evidence of that to the extent they’re wrong, but where they’re right they may be getting it from a system prompt and in some cases are known to be e.g. Claude I was deliberately including that as a way to pass, because fixing this via system prompt best practices is fine
-
Comprehensive Survey of AI Prompting Techniques and Methods
By
–
Systematic Survey of Prompting Techniques: https://
arxiv.org/abs/2406.06608 [77-page PDF]
======
#AI #GenAI #GenerativeAI #PromptEngineering #RAG #LLMs -
Byte Latent Transformer: Patches Match Token Performance with Better Efficiency
By
–
New from Meta FAIR — Byte Latent Transformer: Patches Scale Better Than Tokens introduces BLT, which for the first time, matches tokenization-based LLM performance at scale with significant improvements in inference efficiency & robustness. Paper https://
go.fb.me/w23lmz -
Byte Latent Transformer Repository Available on GitHub
By
–
More in the Byte Latent Transformer repo on GitHub
-
2025 AI Engineering Reading List: Weekly Papers
By
–
Presenting: The 2025 AI Engineering Reading List https://
latent.space/p/2025-papers 1 paper/blog/model family per week for every week of 2025, for you to run paper clubs or binge over the break. -
Why models trained on ChatGPT don’t all excel
By
–
If DeepSeek-V3 is good because it trained on ChatGPT (which of course it did), why isn’t Grok amazing? Why isn’t *every* model amazing? Why spend 95% of compute pre-training a new model (which equals 405B on Pile-test btw) if the secret sauce is ~fOrBiDdEn~DaTa~ in the last 5%?
-
Llama 3.2 Vision Model for Invoice Processing Tasks
By
–
Since you're processing individual invoices, I would recommend trying with Llama 3.2 vision model directly. If there the accuracy is not great, then you can try other methods.
-
Getting Started with RAG for Complex Document Processing
By
–
This is a good place to get started with RAG over complex docs! We'll cover more advanced techniques very soon! Find all the code here: https://
github.com/patchy631/ai-e
ngineering-hub/tree/main/rag-with-dockling
… -
Setting Up LLM and Embedding Model After Knowledge Base
By
–
Once our knowledge base is ready it's time to setup the LLM and embedding model! Check this out