Thanks for the kind words, and I am glad you got lots out of it! Btw in case you feel bored, I added a bunch of bonus material to the GitHub repo recently. For example the conversion to a Llama 3.2 model: https://
github.com/rasbt/LLMs-fro
m-scratch/tree/main/ch05/07_gpt_to_llama
…
@rasbt
-
Bonus Material Added: GPT to Llama 3.2 Model Conversion
By
–
-
Upcoming Blog Article on Multimodal LLMs Coming Tomorrow
By
–
Ha, what a timely question. Spoiler: I meant to post an extensive blog article on multimodal LLMs today but got a bit behind with editing and uploading. But keep an eye out for it tomorrow (
https://
magazine.sebastianraschka.com) -
CPU and GPU Compatibility: Code Automatically Adapts to Available Hardware
By
–
Interesting question. Not sure this is possible. However, this also shouldn't be necessary for the book. The code works on both CPU and GPU machines (the code is written that it automatically uses a compatible GPU if available)
-
BPE Implementations Comparison and Julia Experiments Updates
By
–
Awesome, glad to hear! And I am curious about the Julia experiments, but absolutely no rush 🙂
Btw I also did some quick comparisons with some other BPE implementations here: https://
github.com/rasbt/LLMs-fro
m-scratch/blob/main/ch02/02_bonus_bytepair-encoder/compare-bpe-tiktoken.ipynb
…
(I also have my own custom one that I plan to polish up and add some time…) -
Python and PyTorch prerequisites for machine learning course
By
–
Thanks for your interest! I'd say familiarity with Python programming would be required. Ideally some light machine learning or deep learning background with PyTorch. However, I do have a ~40-page introduction to the essentials (Appendix A), which could get you up to speed quick
-
MacBook Air M3 with 24GB RAM Handles Llama 3.2 Well
By
–
Yeah. But in the meantime, the MacBook Air M3 24 Gb is just fine for Llama 3.2 😛
-
Synthetic Data Generation as Prototype Example, Not Endorsement
By
–
I see it more of a prototypical example of synthetic data gen, not an endorsement of synthetic data gen 😛
-
Anticipation for upcoming early stopping and data retraining discussion
By
–
Great post, looking forward to the next one on early stopping and full data retraining (haha, our hotly debated topic in the past )
-
CodeLlama PR merged with 4-bit quantization inference benchmarks
By
–
The CodeLlama PR just got merged: https://
github.com/Lightning-AI/l
it-gpt/pull/472
… When I tried it with bnb's 4-bit Normal Float quantization, the 34B Instruct and Python variants used about 20 Gb for inference: -
Efficient LLM Training QLoRA Llama 2 Resource Optimization
By
–
Depends on your settings. But if you limit the context size to like 2048 (like in the NeurIPS competition) and use a microbatch size of 1 with gradient accumulation and qlora with llama 2 7B, that’s approx 20 GB RAM and shouldn’t take too long, maybe an hour.