The Ultiamte Step-By-Step LLM Engineering Projects Roadmap – Build a tokenizer
– Learn embeddings
– Implement RoPE / ALiBi
– Hand-wire attention
– Build MHA
– Build a Transformer block
– Train a mini-former
– Compare objectives
– Build sampling
– Speculative decoding
– KV cache
LLM Engineering Roadmap Covers Tokenization, Attention, Sampling and Speculative Decoding
By
–
