GPT-2 is the "hello world" of LLMs I think (there must be a better analogy… err MOS 6502? xv6?), so that's why I started there. And it has a proper paper, weights released and available, and a lot is known about it. At this point it is an artifact of historical significance.
OPEN SOURCE
-
GPT-2 Weight Initialization and Fine-tuning Strategy
By
–
atm we're doing init from gpt-2 weights and finetuning. this was very useful for debugging and when the code was slower. there is no code yet to init from scratch, so no code to warmup the lr etc. should be a very short addition though.
-
Lack of Public Documentation in AI Implementation Practices
By
–
That sounds like it would work in theory, I'd love to see documented examples with live demos of people doing that in practice I get the impression that people who do this rarely share their results publicly, which is a shame
-
PyTorch Support and llm.c Project Development Status
By
–
Part agree! I love PyTorch ofc. But also llm.c is a ~2 week old project that is worked on by ~3 people as a hobby in spare time.
-
torch.compile uses Triton kernels under the hood for optimization
By
–
So if you're using torch.compile you're already using a lot of triton under the hood, afaik PyTorch picks and chooses whether to call cuda kernels or triton for different ops / settings. Triton is really awesome, but of course you're staying in the Python / torch universe. Which
-
llm.c Matches PyTorch Performance Training GPT-2 on GPU
By
–
llm.c update: Our single file of 2,000 ~clean lines of C/CUDA code now trains GPT-2 (124M) on GPU at speeds ~matching PyTorch (fp32, no flash attention) https://
github.com/karpathy/llm.c
/blob/master/train_gpt2.cu
… On my A100 I'm seeing 78ms/iter for llm.c and 80ms/iter for PyTorch. Keeping in mind this is fp32, -
Free GPT-4 and Llama 3 alternatives to ChatGPT
By
–
There really is no reason anyone should be using free ChatGPT-3.5 anymore. Not only can you still get GPT-4 access for free through Microsoft Copilot (in creative mode), but if you want a GPT-3.5 class model, Llama 3 is much better & free (for now), here: https://
meta.ai -
Open Models Drive Rapid AI Capability Improvements and Speed
By
–
Because anyone can work with them, open models are likely to improve very quickly, creating a lot of capabilities focused on factors ranging from speed to costs.
— Ethan Mollick (@emollick) 19 avril 2024
Here is the new Llama 3 70B being served by Groq (with a q) at 224 tokens/second. This is real-time of me using it. pic.twitter.com/L6i6T6OBbWBecause anyone can work with them, open models are likely to improve very quickly, creating a lot of capabilities focused on factors ranging from speed to costs. Here is the new Llama 3 70B being served by Groq (with a q) at 224 tokens/second. This is real-time of me using it.
-
LangChain Weekly Release: Evaluations, Tool Calling, Monitoring
By
–
LangChain Release Notes, week of 4/15 Evaluations video series Standardized tool calling in LangChain Production monitoring and automations in LangSmith New RAG From Scratch videos
Community created content! Read it all here: https://
blog.langchain.dev/week-of-4-15-l
angchain-release-notes/
… -
Open-Source Model Beats Claude 3 Opus at 300 Tokens Per Second
By
–
We now have an open-source model that is beating Claude 3 Opus…
— Matt Shumer (@mattshumer_) 19 avril 2024
being served at nearly **300 tokens per second** on @GroqInc.
The applications built off of this tech will be nothing short of revolutionary. pic.twitter.com/v934g0rwU5We now have an open-source model that is beating Claude 3 Opus… being served at nearly **300 tokens per second** on @GroqInc
. The applications built off of this tech will be nothing short of revolutionary.