Victor MARTIN ~ Understanding Automatic Differentiation in 30 lines of Python https://
bit.ly/3t0ejMg #AI #MachineLearning #DeepLearning #LLMs #DataScience
CODE
-
Automatic Differentiation Implementation Guide in Python
By
–
-
GQA Optimization: K and V Attention Projections Explained
By
–
Ah well it's certainly at least done way better than the others at finding something relevant! However it looks like this article might be mistaken in saying the 4 attention projections are the same. IIRC the GQA optimization only applies to K and V, which we see here:
-
Clarifying parameter counts in 7B language models
By
–
That seems wrong? It doesn't ask which size model, but even 7b has way more params than that, and if it's talking about a single 7b layer it's not right either.
-
GPT-4 Long Context Recall: Needle in Haystack Analysis
By
–
Pressure Testing GPT-4-128k With Long Context Recall Some awesome work by @GregKamradt to evaluate performance of GPT-4 when passing it long context He did a "needle in a haystack" analysis to find how it performed at recalling facts – and open sourced the code! Check out
-
Open Source Definition Nuances: Licenses Governance Expertise
By
–
FWIW I think the take on the nuance of “open” is actually a pretty genuine one. What someone means by “open source” varies widely. You can have: – difference licenses – difference governance models – different levels of technical expertise required to understand something
-
UniSim: Learning Interactive Real-World AI Simulators
By
–
UniSim: Learning Interactive Real-World Simulators https://
bit.ly/3QxKrjL
#AI #MachineLearning #DeepLearning #LLMs #DataScience -
No Free Lunch in Convex Optimization: Momentum Limitations
By
–
lol I received several messages telling me to learn more about optimization so I might have to do that and think about it more my thought process was that there's no free lunch in convex optimization so even if eg momentum works in some cases, it doesn't work in others, which is
-
Ballmer’s GPL Cancer Comment: Open Source History
By
–
Ballmer did say that the GPL (used by the Linux kernel) was a *cancer".
-
New ControlNet Preprocessors Model Available on Replicate
By
–
I’ve added a small new model to Replicate that lets you run a lot of the ControlNet preprocessors: https://
replicate.com/fofr/controlne
t-preprocessors
… It uses Patrick von Platen’s very helpful controlnet-aux package. Thanks for the heads up @DigThatData -
Using curated tag lists instead of auto-generated wacky tags
By
–
Yeh we actually have that but it makes up such wacky tags, so will make it pull from a list