RAG can handle all sorts of different questions, it's really appropriate for extracting information from a document. An interesting question is "when the question requires using more many distant elements from doc, is RAG still relevant?"
LLMS
-
RAG retrieval failures due to random variations, need more tests
By
–
Well spotted!
Both variations (blue and green ) are also due to random variations. For instance the RAG fails to retrieve relevant snippets on 1 single example. The tests I ran were not numerous enough yet to smooth these variations out. But maybe I'll run more. -
RAG system reduces token input from 128k to 2k
By
–
Exactly!
To complement on the end: thanks to the RAG system, the model was fed around 2k tokens each time, down from the 128k tokens of the original document. -
Auto-regressive Prediction Limits: Video vs Language Models
By
–
Thanks, Chris. Auto-regressive prediction has failed for video in the past but not obviously for language. @ylecun has emphasised this drift, but I wonder how much (1) redundancy and (2) discreteness can provide self-correction.
-
Building Retrieval Augmented Generation Chatbot with PDFs
By
–
Building a Retrieval Augmented Generation Chatbot If you are looking to learn how to build an end-to-end app to allows users to talk with their PDFs, check out this from @sophiamyang It covers: Ingestion
Prompting
Frontend Blog: https://
sophiamyang.medium.com/building-a-ret
rieval-augmented-generation-chatbot-d567a24fcd14
… -
Six months freelance Data Science: practical tips and insights
By
–
6 months as a Data Science freelance – some tips https://
bit.ly/40UpSRQ
#AI #MachineLearning #DeepLearning #LLMs #DataScience -
Discreteness Redundancy Self-Correction Images Language Genes
By
–
And what role does discreteness play here? Images are also redundant but maybe not in the same way as English. So I’m not sure which accommodates better self-correction. I’m thinking of genes too. Clear answers appreciated.
-
Redundancy and Error Correction in Autoregressive Language Model Learning
By
–
Redundancy in language, eg “I am Sam” vs “am Sam”, makes communication robust to errors. Is there a crisp argument for why this error-correction helps in learning auto-regressive language models? The intuition seems right but I’d love to see a formal proof or experiment.
-
7 Essential Things to Know About LLM Fine-Tuning
By
–
ICYMI: We recently hosted an interactive discussion with our #LLM experts on 7 Things You Need to Know About #Finetuning. Here's the session recording: https://
pbase.ai/3sRBVDd. And, if you're interested, try fine-tuning and serving LLMs for free: https://
pbase.ai/3RpF5Hy. -
OpenAI Launches Public API Changelog for Developers
By
–
Small but useful update for @OpenAI devs: We now have a public API changelog where you can keep track of how our API is evolving over time. Check it out :