There's some REALLY cool improvements here My two favorites: Automatic Metadata Indexing and support for Redis Filter Expression Language
CODE
-
HumanEval Benchmark: Assessing AI Code Generation Capabilities
By
–
Time to read again Loubna’s nice post from last week diving in the HumanEval benchmark
-
RAG Technique: Solving Modern LLM Limitations Without Fine-Tuning
By
–
Let’s dive in to one of the most-talked about AI techniques—retrieval augmented generation (RAG)— that developers are using as a precursor to fine-tuning. It helps solve the recent issue of modern LLMs while being not-really-that-hard to implement.
-
Retrieval Systems Fetch Parent Documents for Enhanced Context
By
–
Usually retrieval system then fetch each chunk individually. The idea behind this is you fetch the parent document they come from (for more context)
-
RAG: Retrieval Augmented Generation Explained for Enterprises
By
–
RAG stands for retrieval augmented generation, which enables companies to fetch important data relative to a query in a prompt in order to improve its results.
-
RAG Emerges as Developer Priority for Model Performance Optimization
By
–
Among all the developers, experts, and sources I talk to, no subject comes up more than RAG. Developers are increasingly working with it as a precursor to fine-tuning to squeeze more performance out of less-powerful models.
-
Pretraining Llama 2 with New Data: Tutorial and Approach
By
–
Instead of training from scratch, you could take the existing Llama 2 base model and pretrain it for a few more epochs on new data and see how it performs. I set up a tutorial here the other day (you may want to swap the dataset depending on your usecase): https://
github.com/Lightning-AI/l
it-gpt/blob/main/tutorials/pretrain_redpajama.md
… -
Summer LLM Developments: Llama 2, CodeLlama, and GPT-4
By
–
Llama 2, CodeLlama, and leaked GPT-4 details. Here's my new write-up on the noteworthy developments around LLMs of this summer so far:
-
Implementing AI: accessibility and RLHF resources
By
–
Still a great list. Today I would add to dive as soon as possible in implementing something yourself since recent AI developments have become so accessible. We still need more good book/ressources on RLHF, maybe @natolambert or @_lewtun will fill this gap soon 🙂
-
Half of Neural Network Unused Yet Produces Reasonable Outputs
By
–
that feeling when you find a bug where half of your neural network wasn't even being used, and yet its outputs still look extremely reasonable