If you want to read about them from the hardware perspective, read these three papers: https://
arxiv.org/abs/1704.04760
(about TPU v1, the one that started it off) https://
arxiv.org/abs/2304.01433 (about TPU v4 and its optical interconnect) https://
gwern.net/doc/ai/scaling
/hardware/2021-jouppi.pdf
… (ten lessons)
TPU Hardware Architecture: From v1 to v4 Optical Interconnect
By
–