This is very important in Deep Learning: it allows calculating derivatives of highly complex neural networks, composed of stacked layers of simple functions. Gradient descent is used to update the neural net weights, allowing it to learn from data and perform complex tasks.
AI
-
LLM Collaboration and Chain Rules Innovation
By
–
The thread above was, of course, written in collaboration with one of our powerful Large Language Models. Long live Chain Rules!
-
Token Probability in Language Models Explained
By
–
By smaller events here, we refer to the probability of a token, given past tokens, p(c|ab). In probabilistic language modeling, a “token” is a single unit of text, like a word or part of a word. Modern language models consider a vocabulary size of ~100K tokens.
-
Chain Rule Reduces Token Generation Complexity Exponentially
By
–
To generate a sequence of 1000 tokens requires an insane 100K^1000 = 10^5000 choices. That’s a lot more than the estimated number of atoms in the universe, 10^82! With the chain rule the number of possible choices is "only" 100K * 1000 = 100M, a much more manageable number.
-
Chain Rules: Probability and Calculus in Machine Learning
By
–
Chain Rule(s) rules! Appreciation thread of one of the most interesting coincidences in machine learning. Two rules, both named "Chain Rule", happen to be absolutely critical to recent advances in ML & AI. A on the Chain Rule of Probability & the Chain Rule of Calculus
-
Chain Rule of Probability Powers Large Language Models
By
–
The Chain Rule of Probability is a powerful tool behind recent advances in Large Language Models. By multiplying together the probabilities of many smaller events, we can compute the probability of a complex event made up of those smaller events.
p(abc) = p(c|ab) * p(b|a) * p(a) -
AI Progress Accelerates: Balancing Innovation with Responsible Deployment
By
–
Most amazing fact about AI is that even though it’s starting to feel impressive, a year from now we’ll look back fondly on the AI that exists today as quaint & antiquated. Equal cause for excitement and deliberative caution — important to get the tech and its deployment right.
-
DirectoryLoader LangChain loads multiple documents with sources
By
–
Loading multiple documents and associating the sources is a great idea! And just happens there’s a “DirectoryLoader” which does exactly that https://
langchain.readthedocs.io/en/latest/modu
les/document_loaders/examples/directory_loader.html
… -
Ghost Cites and ChatGPT: Academic Citation Crisis Worsens
By
–
This is a classic story about an article that was never written yet became widely cited. "Ghost cites" were always problem in academia. But it's about to get much, much worse. This is how Chat GPT summarizes this famously non-existent article:
-
Browser Extension Leverages ChatGPT for Enhanced User Productivity
By
–
Source: https://
reddit.com/r/MachineLearn
ing/comments/10ujsk5/p_i_made_a_browser_extension_that_uses_chatgpt_to/
…