LLMS

→ View original post on X — @huggingface

New Architecture Combines Transformers and RNNs for Faster Inference

By

@huggingface

–

15 May 2023 12h17

The architecture takes the best from transformer-based models and RNNs to provide faster inference and theoretical infinite context length

→ View original post on X — @huggingface

RWKV Models Now Integrated in Transformers Library

By

@huggingface

–

15 May 2023 12h17

The first RNN in transformers! Announcing the integration of RWKV models in transformers with @BlinkDL_AI and RWKV community!
RWKV is an attention free model that combines the best from RNNs and transformers.
Learn more about the model in this blogpost: https://
huggingface.co/blog/rwkv

→ View original post on X — @thom_wolf

StarCoder 15B Outperforms Larger Models on Reasoning Tasks

By

@thom_wolf

–

15 May 2023 8h41

We saw that with StarCoder in @BigCodeProject as well: being trained on code the 15B StarCoder is beating much larger models on HELM synthetic reasoning and logic tasks, like cohere command beta 52B, Anthropic-LM v4 52B, Aleph-Alpha Luminous Supreme 70B or OPT 175B

→ View original post on X — @waitin4agi_

Best Methods for Training AI Personality and Virtual Companions

By

@waitin4agi_

–

15 May 2023 8h17

The best way to mimic personality is probably not fine tuning GPT-4 (which we did in our last Unreal Engine demo) . For those wondering what the best way to train friend's personality, virtual girlfriends, I would recommend using:

Fine-tuning Flan collection with T5X and SeqIO

By

–

15 May 2023 8h09

put the flan collection into seqio and finetune it in t5x? even i don't have access to this code anymore. im not sure if the seqio task registries have been open sourced.

→ View original post on X — @wadhwaniai

Transformer Models Demystified: Reshaping AI Landscape

By

@wadhwaniai

–

15 May 2023 7h54

Our Senior ML Scientist @MakarandTapaswi demystifies the inner workings of transformer models, the 'T' in #ChatGPT, in his article for @the_hindu
, and explores how these models are reshaping #AI

FLAN-T5 and FLAN-UL2 Outperform Vanilla Models

By

–

15 May 2023 5h38

they are orthogonal concepts. flan-{t5/ul2} is almost always better than the vanilla model.

UL2 Fine-tuning Outperforms T5 in Most Cases

By

–

15 May 2023 5h01

Fwiw, fine-tuning ul2 is almost always better than t5.

540 Billion Parameters: Scaling Language Models

By

–

15 May 2023 3h53

Hahahah, 540 is already a reserved number in my head that if you prompt me with "540" I will continue by saying "billion parameters" almost instinctively.