MACHINE LEARNING

→ View original post on X — @_akhaliq

LLM-Blender: Ensembling Large Language Models with Pairwise Ranking

By

@_akhaliq

–

06 June 2023 3h04

LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion paper page: https://
huggingface.co/papers/2306.02
561
… present LLM-Blender, an ensembling framework designed to attain consistently superior performance by leveraging the diverse strengths of multiple

→ View original post on X — @miketamir

Explore 4 New Arts and AI Experiments

By

@miketamir

–

06 June 2023 3h01

Try 4 new Arts and AI experiments https://
bit.ly/45iGxjP #AI #MachineLearning #DeepLearning #LLMs #DataScience

→ View original post on X — @_akhaliq

Orca: 13B Model Learning from GPT-4 Explanation Traces

By

@_akhaliq

–

06 June 2023 2h57

Orca: Progressive Learning from Complex Explanation Traces of GPT-4 paper page: https://
huggingface.co/papers/2306.02
707
… develop Orca, a 13-billion parameter model that learns to imitate the reasoning process of LFMs. Orca learns from rich signals from GPT-4 including explanation traces;

GPU Memory Capacity vs Mixed-Precision Training Support Trade-offs

By

–

06 June 2023 0h32

Hah, yes, same. But it's basically 1 GPU with multiple cores. I am more excited about the 192 Gb GPU memory here. But then, would it support mixed-precision training and bfloat 16? Probably no. So that's actually worse than than an A100 with 48 Gb for example.

→ View original post on X — @terence_mills

Generative AI Integration in Amazon Web Services Platform

By

@terence_mills

–

06 June 2023 0h30

Generative AI comes to Amazon Web Services
#AI #AIio #BigData #ML #NLU #Futureofwork @Kevin_Jackson @sallyeaves @schmarzo @kdnuggets @akwyz @BrettKing @TechNative @MiaD http://
ow.ly/cy6430svtqJ

CORN Method for Finetuning LLMs on Prediction Tasks

By

–

06 June 2023 0h04

Oh, and I should have emphasized that you can use our CORN (Conditional Ordinal Regression for Neural Networks) method to finetune your favorite LLM for prediction tasks, or course! An example for TripAdvisor customer reviews here: https://
github.com/Raschka-resear
ch-group/coral-pytorch/blob/main/docs/tutorials/pytorch_lightning/distilbert-corn-tripadvisor.ipynb
…

Excessive Data Duplication and Multiple Epochs Lead to Overfitting

By

–

05 June 2023 21h37

And another recent one. That one shows that too much duplicated data (here, ~ training for many epochs) leads to overfitting:

5 June 2023

Pythia Paper Findings on Model Architecture Differences

By

–

05 June 2023 21h35

In the Pythia paper, they found that it doesn’t really make a difference if I recall correctly: https://
arxiv.org/abs/2304.01373 @BlancheMinerva may have some additional insights