Here we go. Llama2-7B at 30 tokens per second ON CHIP. Give it 2 more cycles and we’ll see the larger models on device at these speeds. LLMs will be as omnipresent as calculators, simple maths.
HARDWARE
-
Altgreen’s Eco-Friendly Electric Three-Wheeler Solution for India
By
–
@AltigreenL introduces a more eco-friendly and cost-effective solution for deliveries in India. This electric three-wheeler uses only 12 units of electricity per charge #ModelBasedDesign #ElectricVehicles #Simuliink
-
Dump Google as AI Transforms Economy, Says Analyst
By
–
Sell-side most interesting: Dump Google, and AI pervades the economy Time do dump Alphabet as economic clouds darken, says Monness Crespi Hardt’s Brian White, TI and ON Semiconductor grapple with weak industrial chip markets, and vast sectors of the economy will move to
-
3D Printing Enables Functional Biological Body Parts for Prosthetics
By
–
Los avances en impresión 3D son enormes y no queda mucho para que podamos crear partes del cuerpo humano con esta tecnología, completamente funcionales y con material biológico.
— Juan Merodio (@juanmerodio) 25 octobre 2023
Esto abre un nuevo horizonte para todas las personas con prótesis que podrán ver cómo su vida… pic.twitter.com/jBorNAI9bhLos avances en impresión 3D son enormes y no queda mucho para que podamos crear partes del cuerpo humano con esta tecnología, completamente funcionales y con material biológico. Esto abre un nuevo horizonte para todas las personas con prótesis que podrán ver cómo su vida
-
A40 GPU achieves 34.51 iterations per second performance
By
–
34.51it/s on A40
1 step That's like 29ms? -
A40 GPU Generates 100 512×512 Images in 27 Seconds
By
–
I just generated 100 batched 512×512 images in 27 seconds on an A40 using a latent consistency model I think it can go faster.
-
When Can We Use Pure BF16 in AI Models?
By
–
Of course. But lots of people have observed the same thing in other contexts, and it's what we'd expect based on first principles. To me, the question isn't: when can't we use pure bf16, but when can we!
-
Distributed GPU Processing: File Synchronization and Aggregation
By
–
because in distributed mode, each GPU runs its own process. so this code will run simultaneously multiple times, once per GPU. so in this setting, each GPU writes a file, waits for everyone, then aggregates all the files, waits again, then deletes its own file 🙂
-
RISC-V Space Technology Event in Denver Next Week
By
–
We will be in Denver next week talking all things #riscv in #space, alongside our partners @BrekerSystems
, @Codasip
, @arteris_noc
, RISC-AI,
@ImperasSoftware, @cycuity and @Synopsys for an event exploring the intersection of #riscv, electronics design and space. Sign up here -
AI Dependency on Human Infrastructure and Maintenance
By
–
AI require electrical power, hardware maintenance, software updates, and many other things that they can't get without humans.