AI HARDWARE

→ View original post on X — @fofrai

A40 GPU achieves 34.51 iterations per second performance

By

@fofrai

–

25 October 2023 18h51

34.51it/s on A40
1 step That's like 29ms?

→ View original post on X — @fofrai

A40 GPU Generates 100 512×512 Images in 27 Seconds

By

@fofrai

–

25 October 2023 12h12

I just generated 100 batched 512×512 images in 27 seconds on an A40 using a latent consistency model I think it can go faster.

When Can We Use Pure BF16 in AI Models?

By

–

25 October 2023 8h27

Of course. But lots of people have observed the same thing in other contexts, and it's what we'd expect based on first principles. To me, the question isn't: when can't we use pure bf16, but when can we!

FP32 vs BF16 vs FP16 Mixed Precision Training Options

By

–

25 October 2023 3h43

The main options are fp32 (or tf32), bf16, or fp16+mixed prec afaik.

BF16 vs FP16 Training Performance Comparison Analysis

By

–

25 October 2023 3h41

There was some comparison of training performance by @StasBekman recently that showed bf16 non-mixed training was quite a bit worse. Which I'd expect, because bf16 has much less precision than even fp16.

→ View original post on X — @groqinc

Groq Silver Sponsor ACTIAC Imagine Nation Conference 2023

By

@groqinc

–

25 October 2023 1h17

@GroqInc is a proud Silver Sponsor of the @ACTIAC Imagine Nation #ELC23. Email contact@groq.com to set up a meeting to discuss strategies for advancing #AI and the implications of shifting from an inference chip to an LPU™ system.

BF16 Benefits Beyond Loss Scaling in Mixed Precision

By

–

25 October 2023 0h17

Other than avoiding loss scaling, are there other benefits to using bf16? It still seems to require mixed precision afaict, right?