He can sit back while startups develop and host the best open source LLMs on AWS. Solid passive income strategy.
@rasbt
-
Nvidia Chips as a Subscription Service Model
By
–
Actually I think that’s smart. This way you don’t have to deal with reselling and recycling old chips every few years. Nvidia chips should be offered as a subscription 😀
-
Training Materials Using Project Gutenberg Public Domain Corpus
By
–
Yes! The bonus materials include training on the Project Gutenberg public domain book corpus. I don’t want to go beyond that though and curate other datasets because of copyright concerns. However, you could eg use the FineWeb dataset which is available from hugging face.
-
Reinforcement Pre-training Already Explored in Earlier Research
By
–
Have to look into this one more, but it’s already been a thing; eg “Reinforcement Pre-training” from earlier this summer
-
Missing Comparisons with GPT-5-Codex and Gemini 2.5 Pro Models
By
–
Hard to tell given that comparisons with gpt5-codex and gemini-2.5-pro etc are missing
-
Building and Pre-training Neural Network Architecture from Scratch
By
–
We build the architecture from scratch and pre-train it on a small dataset. But we keep the pre-training short to keep it accessible. Then, we load pre-trained weights and further fine-tune it.
-
Stable Diffusion: A Retrospective on Naming and Legacy
By
–
In hindsight, Stable Diffusion was actually a good one. Sure, didn't end well, but the name itself wasn't a bad choice.
-
Major Labs Avoid Fine-tuning Competitors’ Models Due to Company Policy
By
–
Fair. I think a lot of that is because the major labs are commercial labs, and they can’t (due to company policy)/don’t like to finetune competitors’ models
-
Nvidia Nemotron’s Superior Performance Influences Llama Model Improvements
By
–
Historically Nvidia’s Llama Nemotron models always outperformed the official Llama variants, and I am sure the llama team took a look at what they did to improve their own pipeline. I am sure they benefited from this, and this wouldn’t have been possible without open source.
-
Community Fine-tuning Benefits for Model Developers
By
–
I think finetuned models can also be a benefit for the developers. Ie developers may release a base model, and the community applies continued pretraining and finetuning to make a Math, Code, Instruct, Reasoning variant etc so the developers don’t have to.