Confirmed! In addition to improved text generation, GPT-4o can now return multiple images as part of a single response. It appears that an updated system prompt for the DALL·E tool has changed its operational behavior. https://x.com/SamsonWu_AGI/status/1802421281876238354
LLMS
-
Experimenting with LLM Apps and Agents on Mobile Phones
By
–
Very cool @dr_cintas
, I should better try experimenting with LLM apps on phones and maybe running some complex logic via agents. Great tutorial btw -
Table-LLaVa 7B: Multimodal Table Understanding Competing with GPT-4V
By
–
9/ Multimodal Table Understanding – introduces Table-LLaVa 7B, a multimodal LLM for multimodal table understanding; it’s competitive with GPT-4V and significantly outperforms existing MLLMs on multiple benchmarks.
-
LLM Context Window Enhancement: Extending to 256K Tokens
By
–
10/ Consistent Middle Enhancement in LLMs – proposes an approach to tune an LLM to effectively utilize information from the middle part of the context; it first proposes a training-efficient method to extend LLMs to longer context lengths (e.g., 4K -> 256K).
-
Visual Chain of Thought: Sketching Framework for Multimodal LLM Reasoning
By
–
7/ Sketching as a Visual Chain of Thought – a framework that enables a multimodal LLM to access a visual sketchpad and tools to draw on the sketchpad; it can equip a model like GPT-4 with the capability to generate intermediate sketches to reason over complex tasks.
-
Mixture of Memory Experts Reduces LLM Hallucination Tenfold
By
–
8/ Mixture of Memory Experts – proposes an approach to significantly reduce hallucination (10x) by tuning millions of expert adapters (e.g., LoRAs) to learn exact facts and retrieve them from an index at inference time.
-
Transformers Meet Neural Algorithmic Reasoners Hybrid Architecture
By
–
5/ Transformers Meet Neural Algorithmic Reasoners – a new hybrid architecture that enables tokens in the LLM to cross-attend to node embeddings from a GNN-based neural algorithmic reasoner (NAR).
-
Self-Tuning LLMs Improve Knowledge Acquisition from Raw Documents
By
–
6/ Self-Tuning with LLMs – improves an LLM’s ability to effectively acquire new knowledge from raw documents through self-teaching.
-
SelfGoal: LLM Agent Framework for Hierarchical Goal Achievement
By
–
3/ SelfGoal – a framework to enhance an LLM-based agent's capabilities to achieve high-level goals; adaptively breaks down a high-level goal into a tree structure of practical subgoals during interaction with the environment.
-
Mixture-of-Agents: Multi-LLM Collaborative Layer Architecture
By
–
4/ Mixture-of-Agents – an approach that leverages the collective strengths of multiple LLMs through a Mixture-of-Agents methodology; layers are designed with multiple LLM agents and each agent builds on the outputs of other agents in the previous layers.