LLMS

→ View original post on X — @replit

Quadratic AI Training Costs: How Engineers Build Bigger Models Affordably

By

@replit

–

03 July 2023 22h00

Did you know that the cost of training an AI model increases approximately quadratically as you increase the number of tokens and the model size? ML engineers are still building bigger models without breaking the bank. pic.twitter.com/tB7EiintQl
— Replit ⠕ (@Replit) 3 juillet 2023

Did you know that the cost of training an AI model increases approximately quadratically as you increase the number of tokens and the model size? ML engineers are still building bigger models without breaking the bank.

→ View original post on X — @replit

GPU Advances Enable Bigger, Wider LLM Model Architectures

By

@replit

–

03 July 2023 22h00

With newer GPU technologies, the shape of our LLM models will begin to change. Bigger, wider, models are now a possibility.

To learn more watch the full AI panel with @MosaicML’s Chief Scientist @jefrankle in conversation with Replit’s @amasad and @pirroh pic.twitter.com/A4g56a5Ztj
— Replit ⠕ (@Replit) 3 juillet 2023

With newer GPU technologies, the shape of our LLM models will begin to change. Bigger, wider, models are now a possibility. To learn more watch the full AI panel with @MosaicML
’s Chief Scientist @jefrankle in conversation with Replit’s @amasad and @pirroh

Using nn.Linear and nanoGPT for Educational Implementation

By

–

03 July 2023 21h54

Given that you are using nn.Linear, also looking at nanoGPT . (Although here, I guess the reimplementation serves educational purposes)

→ View original post on X — @miketamir

GitHub Guide: Tips for Working with Large Language Models

By

@miketamir

–

03 July 2023 21h01

GitHub – brexhq/prompt-engineering: Tips and tricks for working with Large Language Models like OpenAI's GPT-4. https://bit.ly/46l3eoi #AI #MachineLearning #DeepLearning #LLMs #DataScience

Vision Transformers: Tokenizing Images Like Language Models

By

–

03 July 2023 17h40

In a tweet: similar to LLMs (language transformers) but tokenizing images instead of text:

→ View original post on X — @miketamir

MIT MAS.S68 Course on AI and Machine Learning

By

@miketamir

–

03 July 2023 17h37

MIT MAS.S68! https://
bit.ly/441rwBF #AI #MachineLearning #DeepLearning #LLMs #DataScience

Vision Transformers for Images and LLMs for Text

By

–

03 July 2023 17h13

Oh this post is about vision transformers for images and LLMs for text

Reducing GPU Memory for LLMs and Vision Transformers in PyTorch

By

–

03 July 2023 17h08

One of the big bottlenecks with LLMs & Vision Transformers is GPU memory on consumer devices. Wrote about my favorite techniques for reducing peak memory in PyTorch: https://
lightning.ai/pages/communit
y/tutorial/pytorch-memory-vit-llm/
… Focused on techniques that don't require architecture changes! Suggestions welcome!

ChatGPT’s Knowledge Cutoff Limitation and Synthetic Data Challenges

By

–

03 July 2023 15h57

3/3 Intuitively one major criticism of ChatGPT is it's 2020 knowledge cut-off. If you are only using synthetic data from old models, it'd be challenging to get more contemporary knowledge into the model.