Oof. What a read! https://
forbes.com/sites/kenrickc
ai/2023/06/04/stable-diffusion-emad-mostaque-stability-ai-exaggeration/
…
@rasbt
-
Impressive Read on Stable Diffusion and Stability AI
By
–
-
Stability AI Paper Controversy and Latent Diffusion Discussion
By
–
Saw the discussions regarding Stability AI today, & I am a bit surprised that the paper behind it was allegedly swept under the rug.
I do think it was known that it was based on Latent Diffusion? Covered it back in 2022 in an early Ahead of AI issue: https://
magazine.sebastianraschka.com/p/ahead-of-ai-
4-a-big-year-for-ai
… -
GPU Memory Capacity vs Mixed-Precision Training Support Trade-offs
By
–
Hah, yes, same. But it's basically 1 GPU with multiple cores. I am more excited about the 192 Gb GPU memory here. But then, would it support mixed-precision training and bfloat 16? Probably no. So that's actually worse than than an A100 with 48 Gb for example.
-
CORN Method for Finetuning LLMs on Prediction Tasks
By
–
Oh, and I should have emphasized that you can use our CORN (Conditional Ordinal Regression for Neural Networks) method to finetune your favorite LLM for prediction tasks, or course! An example for TripAdvisor customer reviews here: https://
github.com/Raschka-resear
ch-group/coral-pytorch/blob/main/docs/tutorials/pytorch_lightning/distilbert-corn-tripadvisor.ipynb
… -
Excessive Data Duplication and Multiple Epochs Lead to Overfitting
By
–
And another recent one. That one shows that too much duplicated data (here, ~ training for many epochs) leads to overfitting:
-
Pythia Paper Findings on Model Architecture Differences
By
–
In the Pythia paper, they found that it doesn’t really make a difference if I recall correctly: https://
arxiv.org/abs/2304.01373 @BlancheMinerva may have some additional insights -
New NoPE Paper Released on ArXiv
By
–
Not yet not yet! But you might like this new NoPE paper: https://
arxiv.org/abs/2305.19466 -
Comparison of Rank Consistent Methods: Performance Against CORAL
By
–
And b) it has better predictive performance than other rank consistent methods like CORAL. Here's a side-by-side comparison in terms of how the methods differ.