LLMS

→ View original post on X — @hwchase17

Agent Toolkits Optimized for Newer Chat Models

By

@hwchase17

–

15 June 2023 19h26

A lot of agent toolkits were mostly designed to work with normal LLMs, not chat models But chat models are newer, faster, and generally better We're working on making all our agent toolkits usable with these chat models, s/o @fpingham for doing the first four!

QLoRA, Prefix Tuning, and LoRA: Fine-tuning Techniques Comparison

By

–

15 June 2023 18h04

This was already a lot of coding. I am saving QLoRA for another day! @Yampeleg Regarding prefix tuning/LLaMA-Adapter vs LoRA. The performance is similar, but one advantage of the former is that it allows multimodal inputs (but that's also a post for another day :P)

Language Inclusion in Pretraining and Dataset Swapping

By

–

15 June 2023 18h00

If the language has been included during pretraining, I that shouldn't be a problem (via swapping the dataset). Otherwise, I am not sure:

→ View original post on X — @cerebras

Proprietary Data and LLM Training Strategy for Enterprises

By

@cerebras

–

15 June 2023 17h59

A few topics: * Why proprietary data enables enterprises to build higher quality large language models * Should your organization fine-tune pre-trained models or train from scratch * What are the steps that need to be considered before training your first model

→ View original post on X — @cerebras

Cerebras Tech Talk: Integrating Large Language Models in Enterprise

By

@cerebras

–

15 June 2023 17h58

Join us for a Cerebras Tech Talk! On Thursday, June 22nd at 11:00 AM PT, we will discuss how enterprises can incorporate large language models into their organizations. Register: https://
hubs.li/Q01THKZD0 See below for topics that will be covered in this 30-minute session

Fine-tuning Limitations and Cross-lingual Transfer in Language Models

By

–

15 June 2023 17h52

Interesting point! Next to instruction-finetuning, finetuning is usually for a specific task (e.g., language translation). Not sure how well the model would do if you change source language. I.e., I haven't seen any studies investigating that, yet.

LoRA-trained models show competitive performance with less overfitting

By

–

15 June 2023 17h50

The performance of LoRA-trained models is usually pretty competitive. Sometimes even better than fully finetuned (prob due to less overfitting).
(Table from https://
arxiv.org/abs/2106.09685)

LLaMA-Adapter v2 Adds Trainable RMSNorm and Bias Parameters

By

–

15 June 2023 17h15

Good question. Yes, because LLaMA-Adapter v2 is basically LLaMA-Adapter but it also has trainable RMSNorm parameters and trainable bias units (in FC layers)