I think LLMs are already there. Even the 7b ones. For RAG, the most important thing is search. The better your search is, the better the results. Totally my observation and opinion based on that.
LLMS
-
Two Continua: Retrieval, Synthesis, and Program Execution
By
–
"There is a continuum between reasoning and retrieval" is half-true. There are two continua: one between pure retrieval and novel program synthesis (with many intermediate points that consist of subroutine retrieval + partial synthesis), and one between exact program execution
-
Tools and Methods for Exploring Different AI Models
By
–
What do you currently use for exploring different AI models?
-
Salesforce Launches Einstein Copilot AI Assistant for CRM Applications
By
–
Salesforce introduces new AI assistant, Einstein Copilot, for all its CRM apps https://
bit.ly/3Znmoqi #AI #MachineLearning #DeepLearning #LLMs #DataScience -
LLMs Self-Improvement Through Feedback and Real-Time Learning
By
–
All of these methods explore how LLMs can self-improve based on fine-tuning, implicit human preferences and iterative prompting techniques. LLMs, like humans can take constructive feedback and become better. Now if they can only do it in real-time by just listening /16
-
GPT-4 Self-Improving Code Through Recursive Scaffolding Programs
By
–
The core idea begins with an initial seed 'improver' scaffolding program that utilizes the language model to improve a solution to some downstream task. They demonstrate that GPT-4 is capable of writing code that can call itself to improve itself. /15
-
Self-Taught Optimizer: Language Model for Recursive Solution Enhancement
By
–
Finally, Self-Taught Optimizer (STOP), employs a language model to enhance arbitrary solutions and then applies this recursively to improve itself. /14
-
Promptbreeder Surpasses Chain-of-Thought and Plan-and-Solve Methods
By
–
Promptbreeder outperforms state-of-the-art prompt strategies such as Chain-of-Thought and Plan-and-Solve Prompting on commonly used arithmetic and commonsense reasoning benchmarks. /13
-
Self-Referential LLM Evolution Through Automated Prompt Mutation
By
–
The mutation of these task prompts is governed by mutation prompts that the LLM generates and improves throughout evolution in a self-referential way. Promptbreeder is also improving the mutation prompts that improve these task prompts. /12
-
PromptBreeder: LLM-Driven Self-Improvement Mechanism for Task Prompts
By
–
Another very recent paper introduces PromptBreeder, a general-purpose self-referential self-improvement mechanism that evolves and adapts prompts for a given domain. Driven by an LLM, Promptbreeder mutates a population of task prompts and subsequently evaluates them /11