Oh yes for sure. I took a few Waymo rides a few months ago, and they were awesome.
@rasbt
-
App Development Boom Encourages Building Personal Custom Applications
By
–
I hope this app development boom encourages me/more programmers to build personal & custom apps more often
(just as we have always built our own command-line tools for all kinds of things) -
AI-Generated Code Won’t Reach 90% Adoption Soon, Expert Says
By
–
I think it is more realistic to look at what happened with human-written text. ChatGPT (in 2022) is making a huge difference in how we write. But I wouldn't say that 90% of text is AI-generated (yet). Similarly, I don't think that 90% of code is going to be AI generated in 3-6
-
Self-Driving Cars: Marketing Hype versus Technical Reality
By
–
I remember the timeline of self-driving cars… 90% marketing, 10% reality.
-
Coding Attention Mechanisms: Understanding the Engine of LLMs
By
–
Just uploaded my "Coding Attention Mechanisms" tutorial. A 2h15m session on coding attention mechanisms to understand how the engine of LLMs works: self-attention → parameterized self-attention → causal self-attention → multi-head self-attention
-
Mathematical Similarity in QKV Weight Matrix Implementations
By
–
Yes, they are all mathematically similar. The combined qkv one is interesting as it replaces the 3 separate weight matrices by a single weight matrix multiplication and then splits the result. It's kind of analogous to what the implementation at the end of chapter 3 does with
-
7 Alternative Attention Mechanism Implementations for Chapter 3
By
–
Welcome back, again :). I don't want to ruin your weekend, but based on that chapter, here are 7 more alternative ways to implement the attention mechanism https://
github.com/rasbt/LLMs-fro
m-scratch/blob/main/ch03/02_bonus_efficient-multihead-attention/mha-implementations.ipynb
…
(It's just for fun, pls don't let it hold you back from Ch 4!) -
Test-Time Compute Jargon Alert and Appreciation
By
–
Glad you found it helpful! Haha, and big-time jargon alert for "test-time compute"
-
Reasoning Emergence in LLMs Pre-training and Q&A Data
By
–
You mean that reasoning may emerge from pre-training itself? I think that current-gen LLMs are capable of some reasoning w/o explicit reasoning methodology applied to them. But then, the tricky part is most LLMs nowadays have Q&A data in their pre-training set. And if you think
-
LLM Prompt Classification and Reasoning Toggle Strategies
By
–
Yeah, it's going to be interesting. You can do it either with 1) a 2nd cheap LLM that classifies the prompt as "reasoning" / "non-reasoning" and then modifies the system prompt and/or adds a special token as a toggle
2) do RLFH-style preference tuning to teach the LLM when to