but they are transformer? pls elaborate
@rasbt
-
LLM Architecture Gallery Gets Major Updates and New Diff Tool
By
–
Added lots of improvements to the LLM Architecture Gallery in the last 2 weeks. Imho the coolest one yet: A diff tool many of you were asking for! sebastianraschka.com/llm-arc…
-
Local Models on Diverse Hardware: Market Expectations
By
–
Yes, people will want to run local models on more and different types of hardware. (The question is only if that was already priced into the previous stock valuations, because it was relatively obvious that that's where things are headed.)
-
Memory optimization bottlenecks and quantization trade-offs in LLMs
By
–
I am not sure this will be true, i.e., lower memory demands.
There are some many bottlenecks and opportunities for improvement. If we have better quantization for reducing KV cache sizes via TurboQuant, that just means we will use the memory capacity elsewhere:
– bigger -
Open Source Sustainability Challenges with Large AI Models
By
–
Open source project are usually subsidized by other income and revenue streams (e.g., developers' corporate job and other side projects, companies' products) to keep it sustainable. With open weight models, it's an increasingly large price tag (because of GPU demands), and there
-
Securing Python Packages: Auditing Source Code to Prevent Supply Chain Attacks
By
–
I think the same thing happened with the ctx package a few years back (also through PyPI). Not perfect, but I think the best way to avoid is to: 1. Download a source code snapshot of the package (e.g., from github) 2. Audit it (traditionally manually, but now also LLM
-
Thanks for New Release and MPS Support Improvements
By
–
Nice! thanks for the new release (and also for constantly improving mps support)!
-
Residual Attention and DeepSeek V4 Developments
By
–
Yes, next to residual attention as hot new candidate. And, I am also of course curious to see what DeepSeek V4 is up to haha
-
Cross Attention in Separate Articles on Multi-Modal Models
By
–
Yeah, I have cross attention in my separate articles (the multi-modal and understanding attention from-scratch) ones. This article was focused on text LLMs to keep the scope reasonable.
-
XSA Attention Mechanism: Future Research Candidate for Follow-up Study
By
–
Nice one! XSA from https://
arxiv.org/abs/2603.09078? To keep the scope reasonable, I only focused on those that already made it into the flagship architectures. But maybe that's an interesting candidate for a follow-up on attention research candidates.