5). RATIONALYST – a model for process-supervision of reasoning that enables generalization across diverse reasoning tasks; this process is achieved with pre-training on a collection of 79k rationales from the Pile and a combination of reasoning datasets with minimal human
@dair_ai
-
Architecture Search Framework Optimizes LLM Inference Techniques
By
–
4). Architecture Search Framework for Inference-Time Techniques – introduces a modular framework for building and optimizing LLMs by combining multiple inference-time techniques…
-
LLMs Hide Truthfulness Information in Specific Tokens
By
–
3). LLMs Know More Than They Show – finds that the "truthfulness" information in LLMs is concentrated in specific tokens; this insight can help enhance error detection performance and further mitigate some of these issues.
-
RNNs Revisited: Efficient Parallel Training Without Hidden States
By
–
2). Were RNNs All We Needed? – revisits RNNs and shows that by removing the hidden states from input, forget, and update gates RNNs can be efficiently trained in parallel.
-
Top Machine Learning Papers of the Week
By
–
The Top ML Papers of the Week (Sep 30 – Oct 6): – Movie Gen
– RATIONALYST
– An Analysis of o1-preview
– Were RNNs All We Needed?
– LLMs Know More Than They Show
– Not All LLM Reasoners Are Created Equal Read on for more: -
Comprehensive Survey on Small Language Models: Architectures and Capabilities
By
–
9). Small Language Models Survey – a comprehensive survey on small language models (SLMs) across architectures, training datasets, and training algorithms; analyzes 59 state-of-the-art open-source SLMs and capabilities such as reasoning, in-context learning, maths, and coding.
-
Minstrel: Multi-Agent System with Reflection for Automated Prompt Engineering
By
–
10). Minstrel – a multi-generative agent system with reflection capabilities to automate structural prompt generation; it presents LangGPT, an extensible framework for designing prompts.
-
o1 Preview Model Outperforms GPT-4 in Medical Scenarios
By
–
8). A Preliminary Study of o1 in Medicine – provides a preliminary exploration of the o1-preview model in medical scenarios; shows that o1 surpasses the previous GPT-4 in accuracy by an average of 6.2% and 6.6% across 19 datasets and two newly created complex QA scenarios.
-
RAG Task Categorization Survey: External Data Classification
By
–
7). RAG and Beyond – presents a survey that introduces a RAG task categorization method that helps to classify user queries into four levels according to the type of external data required and the focus of the task.
-
Logic-of-Thought: Propositional Logic Prompting Technique
By
–
6). Logic-of-Thought – proposes a new prompting technique called Logic-of-Thought (LoT) which employs propositional logic to generate and inject expanded logical information from the input context.