Release notes for llm-gpt4all 0.4 are here: https://
github.com/simonw/llm-gpt
4all/releases/tag/0.4
…
OPEN SOURCE
-
llm-gpt4all 0.4 Release Notes Published
By
–
-
Llama 3 8B Now Available via llm-gpt4all Plugin
By
–
I upgraded the llm-gpt4all plugin to support running Llama 3 8B Instruct (thanks, @nomic_ai
) llm install –upgrade llm-gpt4all
llm -m Meta-Llama-3-8B-Instruct 'Write Python code to print 5 great names for a pet pelican' The quantized model is a 4.34GB download, needs 8GB RAM -
Kernel optimization attempts yield minimal performance gains
By
–
added under kernel4 https://
github.com/karpathy/llm.c
/commit/cb791c4ef58d45d58e5af624b0ed41439ac7aeff
…
a bit surprised to only see ~1-2% out of it, which then washes out in training, as the layernorm is not a top-ranking time kernel. Also tried float4 and unrolling but that didn't improve it too much bleh -
PyTorch Pay-Per-Use Model Through Advanced Compiler Technology
By
–
I think some day, projects like PyTorch will make users pay only for the overhead they use. Lots of R&D needed to get there: compiler technology, type systems, various other things. Actually, LuaJIT was really cool in that aspect — one of the most brilliant pieces of
-
Major Data and AI Event: GenAI and Engineering Conference June 10-13
By
–
POV: you would do anything to get into the biggest data and #AI event of the year.
— Databricks (@databricks) 19 avril 2024
Register to hear from leading experts, researchers, and open source contributors about topics like #GenAI and data engineering and streaming.
Join us June 10-13: https://t.co/Rl8WajxnNn pic.twitter.com/x2fNrVOgAJPOV: you would do anything to get into the biggest data and #AI event of the year. Register to hear from leading experts, researchers, and open source contributors about topics like #GenAI and data engineering and streaming. Join us June 10-13: https://
bit.ly/3UlaDA7 -
GPT-2 as the Hello World of Large Language Models
By
–
GPT-2 is the "hello world" of LLMs I think (there must be a better analogy… err MOS 6502? xv6?), so that's why I started there. And it has a proper paper, weights released and available, and a lot is known about it. At this point it is an artifact of historical significance.
-
GPT-2 Weight Initialization and Fine-tuning Strategy
By
–
atm we're doing init from gpt-2 weights and finetuning. this was very useful for debugging and when the code was slower. there is no code yet to init from scratch, so no code to warmup the lr etc. should be a very short addition though.
-
Lack of Public Documentation in AI Implementation Practices
By
–
That sounds like it would work in theory, I'd love to see documented examples with live demos of people doing that in practice I get the impression that people who do this rarely share their results publicly, which is a shame
-
PyTorch Support and llm.c Project Development Status
By
–
Part agree! I love PyTorch ofc. But also llm.c is a ~2 week old project that is worked on by ~3 people as a hobby in spare time.
-
torch.compile uses Triton kernels under the hood for optimization
By
–
So if you're using torch.compile you're already using a lot of triton under the hood, afaik PyTorch picks and chooses whether to call cuda kernels or triton for different ops / settings. Triton is really awesome, but of course you're staying in the Python / torch universe. Which