"Accelerating PyTorch Model Training Using Mixed-Precision and Fully Sharded Data Parallelism" https://
magazine.sebastianraschka.com/p/accelerating
-pytorch-model-training
… A little write-up of a talk I gave last week!
COMPUTING
-
Accelerating PyTorch Training with Mixed-Precision and Fully Sharded Data Parallelism
By
–
-
Metis AIPU: Revolutionary Edge AI Hardware Platform
By
–
🚀 Our very own Paul Neil is in the spotlight! Take a look at this video 🎥 where he talks about our revolutionary Metis AIPU developed for Edge AI applications. Get early access to the Metis AI Platform! https://t.co/66JuRFRT8k#edgeAI #AI #computervision pic.twitter.com/xdahyXpbf4
— Axelera AI (@AxeleraAI) 26 juin 2023Our very own Paul Neil is in the spotlight! Take a look at this video where he talks about our revolutionary Metis AIPU developed for Edge AI applications. Get early access to the Metis AI Platform! http://
axelera.ai/early-access-p
rogram
…
#edgeAI #AI #computervision -
Real-time monitoring pulse beats raw logs aggregates
By
–
Staring at progress meters or raw log streams can be surprisingly useful — you tend to get a sense of the pulse of your job, which can be hard to get from aggregates such as graphs or looking at collected logs after the job completes.
-
Assessing vulnerabilities in AI systems: Process, Network, Infrastructure
By
–
Start with an assessment on process, network, infrastructure and expertise to identify vulnerabilities.
-
Grace Hopper: Computing Pioneer Beyond Age Barriers
By
–
To give context on age, Grace Hopper began computing at 38, completed the first compiler at 46, helped shape COBOL at 53, kept developing COBOL for the Navy in her 70s, retired from the Navy at 80 & then became a consultant for the Digital Equipment Corporation. v/
@cooperx86 -
LOMO: Memory-Efficient Optimizer for Full LLM Parameter Tuning
By
–
5/ LOMO – proposes a new memory-efficient optimizer that combines gradient computation and parameter update in one step; enables tuning the full parameters of an LLM with limited resources.
-
Possessing Titan RTX and Another Nvidia Titan GPU
By
–
i do have a titan rtx but this one was nvidia titan
-
GPT-4 Confirmed 1.7 Trillion Parameters, 10x Larger
By
–
— Latent.Space (@latentspacepod) 25 juin 2023
GPT4 is 8 x 220B params = 1.7 Trillion params https://
x.com/soumithchintal
a/status/1671267150101721090?s=20
… ok I wasn't sure how widely to spread the rumors on GPT-4 but it seems Soumith is also confirming the same so here's the quick clip! so yes, GPT4 is technically 10x the size of GPT3, and all the small -
The Enduring Satisfaction of Merging Code on GitHub
By
–
Something never gets old about pushing the merge button on GitHub.
-
IOPS and Latency Performance Benchmarks for AI Infrastructure
By
–
iops and latency should be quite good but benchmarks are welcome!