Once you have the forward/backward, the rest of it (data loader, Adam update, etc) are mostly trivial. The real fun starts now though: I am now porting this to CUDA layer by layer so that it can be made efficient, perhaps even coming within reasonable fraction of PyTorch, but
OPEN SOURCE
-
Memory Allocation Strategy in LLM Training Implementation
By
–
You can look at the raw training implementation here: https://
github.com/karpathy/llm.c
/blob/master/train_gpt2.c
… You'll see that we allocate all the required memory a single time in the beginning in one large block of 1D memory. From there on during training, no memory gets created or destroyed, so we stay at -
llm.c: Train GPT-2 in Pure C Without Heavy Dependencies
By
–
Have you ever wanted to train LLMs in pure C without 245MB of PyTorch and 107MB of cPython? No? Well now you can! With llm.c: https://
github.com/karpathy/llm.c To start, implements GPT-2 training on CPU/fp32 in only ~1,000 lines of clean code. It compiles and runs instantly, and exactly -
OpenRouter Implementation of AI-Oracle Tool Released
By
–
Thanks to @abdiisan for implementing an OpenRouter version of `ai-oracle`! https://
x.com/abdiisan/statu
/abdiisan/status/1777423304086753725
… -
Serverless ChatGPT RAG with LangChain.js and Azure
By
–
🚀Serverless ChatGPT with RAG using LangChain.js
— LangChain (@LangChain) 8 avril 2024
Great OSS repo that shows how to build a serverless ChatGPT-like experience with Retrieval-Augmented Generation using LangChain.js and Azurehttps://t.co/Bh95rgu7Bb pic.twitter.com/UlLf38DKrbServerless ChatGPT with RAG using LangChain.js Great OSS repo that shows how to build a serverless ChatGPT-like experience with Retrieval-Augmented Generation using LangChain.js and Azure https://
github.com/Azure-Samples/
serverless-chat-langchainjs?tab=readme-ov-file
… -
AI-Oracle: Open-source tool combining Claude, GPT-4, Perplexity
By
–
Open-sourcing `AI-Oracle`.
— Matt Shumer (@mattshumer_) 8 avril 2024
Generates better responses than Claude 3 Opus.
A very simple approach that combines the abilities of Claude 3, GPT-4, and Perplexity to provide better results than any could provide on their own.
Seriously — it's dumb simple.
Notebook in thread: pic.twitter.com/8HQBUEiHoIOpen-sourcing `AI-Oracle`. Generates better responses than Claude 3 Opus. A very simple approach that combines the abilities of Claude 3, GPT-4, and Perplexity to provide better results than any could provide on their own. Seriously — it's dumb simple. Notebook in thread:
-
Free Production-Level Code and MLOps Learning Course
By
–
A 100% free course to Learn How to Write Production level Code and learn MLOps This will teach you everything from Design, Modelling, Testing, Serving Machine Learning Models, and much more for free! 35k+ on Github http://
madewithml.com -
Developer Style Guide for Technical Documentation Contributors
By
–
The goal of this short and sweet style guide is to be "just enough" guidance for technical documentation contributors who aren't regular writers. Thank you @lornajane
! https://
github.com/lornajane/deve
loper-style-guide
… #opensource #techwriting #writethedocs #techcomm -
Commercial Solutions Preferred Over Open Source Tooling
By
–
No amount of tooling will help… people will just use a commercial solution, it’s the easiest path
-
Training LoRAs to Control Stable Diffusion Style Consistency
By
–
It was clear that training needed to be part of the process. SD has way too much entropy to consistently and narrowly represent a single style. Over a few weeks, I trained over a hundred LoRAs on subsets of the source images. LoRA preserves the diversity of base SD while