Parent Document Retriever A new retrieval algorithm that: Creates small chunks (to allow embeddings to have semantic meaning) Fetches the PARENT documents those chunks came from (to capture full context) Parent documents can be either raw documents or larger chunks
LLMS
-
Balancing Chunk Size for Semantic Embeddings in RAG Systems
By
–
There was always a balance between small enough chunks to allow the embeddings to capture the semantic meaning of a chunk, while at the same time wanting them to be long enough to have the full context This helps strike this balance Docs: https://
python.langchain.com/docs/modules/d
ata_connection/retrievers/parent_document_retriever
… -
Expanding Context Windows Improves Semantic Chunk Retrieval
By
–
Can also be thought of as similar to fetching the chunk before/after the most semantically similar chunk This takes advantage of the fact that context windows are getting longer and longer! This would have been less plausible a few months ago
-
Exploring Retrieval Algorithms for LangChain Documentation
By
–
We'd been noodling on this concept for a bit, but spurred into action by @ChrisBoraTweets yesterday: https://
x.com/ChrisBoraTweet
s/status/1689018380584439808?s=20
… What other retrieval algorithms would be good to add? Docs: https://
python.langchain.com/docs/modules/d
ata_connection/retrievers/parent_document_retriever
… -
AI and Power: Ethical Challenges of Automation and Centralization
By
–
https://
bit.ly/2wqaIXz – AI and Power: The Ethical Challenges of Automation, Centralization, and Scale https://
bit.ly/3OiO1eY #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Groq Launches Ultra-Low Latency Generative AI with Llama-2
By
–
Ultra-low latency #generativeAI by @GroqInc is here. Schedule your private demo viewing of Llama-2 70B running on a Groq LPU™ by reaching out to contact@groq.com.
-
CIOs Securing Generative AI Access for Data Science Teams
By
–
Even if an enterprise isn’t ready to consider how generative AI may disrupt their businesses, CIOs need to secure access to #generativeAI models and enable #datascience teams to develop fine-tuned LLMs. Learn more from our own Kjell Carlsson in @CIOonline
: https://
domino.buzz/3qlzkQo -
Giraffe: Addressing Attention Deficit Disorder in Large Language Models
By
–
You can also read more about Giraffe at this link: https://
blog.abacus.ai/blog/2023/07/2
7/treating-attention-deficit-disorder-in-llms/
…. -
Abacus AI Open-Sources Giraffe Long-Context Models
By
–
We are open-sourcing the models and the evaluation datasets and performance experiments. Git repository: https://
github.com/abacusai/Long-
Context
…. You can use the model directly from HuggingFace. 4k version: https://
huggingface.co/abacusai/Giraf
fe-v1-delta-13b-scaled-4
…. 16k version: https://
huggingface.co/abacusai/Giraf
fe-v1-delta-13b-scaled-16
…. -
ReLoRA Paper: Exploring LoRA for LLM Pretraining Beyond Finetuning
By
–
Just catching up with the ReLoRA paper (
https://
arxiv.org/abs/2307.05695) that explores whether LoRA can be used for pretraining LLMs (vs finetuning). Looks promising! Caveat: they pretrained models up to 350 M parameters (the smallest Llama model is 7 B parameters, for comparison)