3/ ToolLLM – enables LLMs to interact with 16000 real-world APIs; allows data preparation, training, and evaluation; the authors claim that one of their models, ToolLLaMA, has reached the performance of ChatGPT (turbo-16k) in tool use.
@dair_ai
-
Skeleton-of-Thought: Parallel API Calls Strategy
By
–
4/ Skeleton-of-Thought – proposes a prompting strategy that firsts generate an answer skeleton and then performs parallel API calls to generate the content of each skeleton point; reports quality improvements in addition to speed-up of up to 2.39x.
-
FLASK: Fine-Grained LLM Evaluation Framework for Alignment Skills
By
–
10/ FLASK – proposes fine-grained evaluation for LLMs based on a range of alignment skill sets; involves 12 skills and can help to provide a holistic view of a model’s performance depending on skill, domain, and level of difficulty.
-
Retentive Network: Foundation Architecture Improving LLM Training Efficiency
By
–
7/ Retentive Network – a foundation architecture for LLMs to improve training efficiency and inference; adapts retention mechanism for sequence modeling that support parallel representation, recurrent representations & chunkwise recurrent representation.
-
Training Dense Retrievers for LLM In-Context Examples
By
–
9/ Retrieve In-Context Example for LLMs – presents a framework to iteratively train dense retrievers to identify high-quality in-context examples for LLMs.
-
Meta-Transformer: Unified Learning Across 12 AI Modalities
By
–
8/ Meta-Transformer – performs unified learning across 12 modalities; supports tasks like fundamental perception (text, image, point cloud, audio, video), application (X-Ray, infrared, hyperspectral, & IMU), and data mining (graph, tabular, & time-series)
-
LLM Challenges: Brittleness, Evaluation, and Experimental Design
By
–
6/ Challenges & Application of LLMs – summarizes a comprehensive list of challenges when working with LLMs that range from brittle evaluations to prompt brittleness to a lack of robust experimental designs.
-
Meta releases Llama 2 foundation models from 7B to 70B parameters
By
–
1/ Llama 2 – a collection of pretrained foundational models and fine-tuned chat models ranging in scale from 7B to 70B; Llama 2-Chat is competitive on a range of tasks and shows strong results on safety and helpfulness.
-
Top ML Papers Week: Llama 2, FlashAttention-2, Retentive Networks
By
–
Top ML Papers of the Week (July 17 – July 23): – Llama 2
– FlashAttention-2
– Meta-Transformer
– Retentive Network
– Challenges & Application of LLMs
– How is ChatGPT’s Behavior Changing Over Time?
… -
Multimodal Transformer Foundation Model for Image and Text Generation
By
–
10/ Generative Pretraining in Multimodality – presents a new transformer-based multimodal foundation model to generate images and text in a multimodal context; enables performant multimodal assistants via instruction tuning.