This is the best documentation I could find of the OpenELM training data – it looks like the bulk of it comes from RefinedWeb, RedPajama, The Pile and Dolma https://
github.com/apple/corenet/
blob/main/projects/openelm/README-pretraining.md
…
CODE
-
OpenELM Training Data Sources Documentation
By
–
-
Apple releases CoreNet: open neural network training library
By
–
CoreNet looks to be the parent project – brand new on GitHub as of a few hours ago, "a library for training deep neural networks" Includes:
– OpenELM: a brand new LLM with open training data
– CatLIP: a CLIP-like model with more efficient training? More: https://
github.com/apple/corenet -
Microsoft Researcher Details Special Tokens in Fine-Tuning Process
By
–
Note from Microsoft researcher Gustavo de Rosa concerning their special tokens: "These tokens have not been used during fine-tuning, i.e., they will be removed in the future. We used: , , , and "
-
Files-to-Prompt tool for LLM context processing
By
–
I built my own tool for that: files-to-prompt to chuck all of the files in a directory into a single stream, then I pipe that to my LLM tool and send it to a large context model like Claude 3 Opus to ask questions about it
-
GPT-3 and Copilot Examples for URL Guessing and Testing
By
–
I have a few examples in https://t.co/aymMPeMsdG and https://t.co/n38dNNX8Vw pic.twitter.com/jPEzopNALi
— Simon Willison (@simonw) 23 avril 2024I have a few examples in https://
til.simonwillison.net/gpt3/guessing-
amazon-urls
… and https://
til.simonwillison.net/gpt3/writing-t
est-with-copilot
… -
Browser Automation and API Security in AI Applications
By
–
I assumed that was what it was doing from the start – those apps don't provide user-facing APIs that you could use to automate them, so the only way to implement their demo would be browser automation and asking users for their passwords
-
AI Text Editor Adds Self-Aware Hack Comments Automatically
By
–
I genuinely love it when it adds "this is a hack" comments, my text editor is judging me!
-
Fine-tuning Primer: Guide to Timing, Budget, Data, and Tools
By
–
Check out this informative primer on all things #finetuning from Daniel Porras Reyes at @flybridge
. Topics: When's the right time to fine-tune Budget / resource considerations Prepping data for fine-tuning Choosing the right tool Check it out: https://
pbase.ai/3JxpMI7 -
AI Model Context Mechanism with Debug Tool Visibility
By
–
Yeah, it has a very neat mechanism where it includes likely snippets from nearby files in the context when it completes – you can run debug tools to see exactly what it's doing
-
Mastering AI coding assistance through practice and proper prompting
By
–
I've also gotten really good at it. I think people who dislike it must have given up too early. You have to learn what to expect, what works, what doesn't work, how to position your cursor (e.g. to not get distracting suggestions), and how to prompt it well via code/comments