Attention is not all you need. Without positional encoding, a transformer would treat a context as a bag of words.
LLMS
-
Gato Architecture: The Future of Multimodal AI Models?
By
–
Meta Chameleon and Gemini have also adopted the Gato ( https://
arxiv.org/pdf/2205.06175 ) architecture. Is this going to be the ultimate approach for MIMO (multimodal input multimodal output models) or is there something else we should be trying? There’s been great progress in scaling -
Intentional AI Development and Gradual Capability Emergence
By
–
– People are trying to build it on purpose
– ChatGPT didn't need to start talking "suddenly" to start talking at some point -
MLflow-Giskard Integration for LLM Testing and Vulnerability Detection
By
–
The MLflow-Giskard integration offers a great solution for testing and validating LLM outputs. MLflow's evaluation API + Giskard's automatic vulnerability detection for LLMs is game-changing. See it in action https://
dbricks.co/4bEoPKl -
User impressions on GPT-4o capabilities
By
–
GPT-4o is insane The capabilities are endless. Quite amazing.
-
Claude 3 Opus Now Available on Vertex AI Platform
By
–
Our most intelligent model, Claude 3 Opus, is now generally available on Vertex AI.
-
Vote for Top 3 Ideas Using 30,000 Tokens Per Second
By
–
A few days ago we asked, "What could you do with 30,000 tokens per second input speed?" We got a bunch of responses. Please vote to select 3 winners. https://
hubs.la/Q02z5P7R0 -
SμPar improves LLM pretraining loss over SP and μP
By
–
(6/n) Applying SμPar to pretraining a 610M parameter LLM significantly improves loss over SP and μP models due to improved HP tuning.
-
Sparse Iso-Parameter Scaling Achieves 87.5% Sparsity
By
–
(7/n) We test Sparse Iso-Parameter Scaling, where width and sparsity are increased, holding n_params constant. Using the standard practice (SP + dense optimal HPs), one would conclude only up to 50% sparsity help. Instead using SμPar allows an 87.5% sparse model to match dense
-
Create AI Agents Without Technical Skills for $10/Month
By
–
Develop AI agents without being technical using English.
— Abacus.AI (@abacusai) 31 mai 2024
For $10/month, ChatLLM Teams gives you access to all LLM chatbots, generates code, and creates powerful AI agents!https://t.co/5QdaueQUqf pic.twitter.com/QA3gPLwdooDevelop AI agents without being technical using English. For $10/month, ChatLLM Teams gives you access to all LLM chatbots, generates code, and creates powerful AI agents! https://
chatllm.abacus.ai