> Instruct beats Claude 3.5 Sonnet in Aider bench presumably they have API access (not sure if v3 preview is on API yet – maybe they got private access if not)
LLMS
-
ARC Grids as Token Sequences: Why VLMs Struggle with Processing
By
–
Fundamentally it's because ARC grids aren't images and thus VLMs can't make sense of them. They're 2D grids of tokens. Some people use 2D native transformers to process them, with good results (2D position embedding, or 2D attention), but a flattened sequence is actually a very
-
DeepSeek-V2.5-1210 Model Comparison Analysis
By
–
I'm comparing w/ https://
huggingface.co/deepseek-ai/De
epSeek-V2.5-1210
… -
MoE Architecture Changes: Softmax to Sigmoid Gate Function
By
–
Dug a bit more in to the modelling code (v2 vs v3), here are the key changes: > MoE gate function changed from softmax (v2) → sigmoid (v3)
> New Top-k Selection method `noaux_tc`
> Added e_score_correction_bias for better expert selection or even training -
Llama v3 vs v2: Key Architecture Configuration Differences
By
–
Dug into the config files a bit, key differences (according to the config files) v2 vs v3: vocab_size: v2: 102400
v3: 129280 hidden_size:
v2: 4096
v3: 7168 intermediate_size:
v2: 11008
v3: 18432 num_hidden_layers:
v2: 30
v3: 61 num_attention_heads:
v2: 32
v3: 128 -
Christmas Break: Explore AI Content on GroqCloud
By
–
Need a break from family on Christmas Day? Find a quiet space to read & check out https://
hubs.la/Q02_Fpnq0.
Here's a sampling:
– Llama-3-1-70b Speculative Decoding on GroqCloud
– Understanding AI 101: What is Inference in ML & AI Applications?
– The Five Future Stages of GenAI -
DPO Research Paper Insights for Large Language Models
By
–
Putting effort in reading LLM research papers gives you so much insight into how really massive models work. DPO method paper: https://
arxiv.org/pdf/2305.18290 -
5 prompts to succeed in your job interview with ChatGPT
By
–
ChatGPT helped my friend prepare for her interview in minutes and she got the job. Here are 5 prompts that will help you get the job:
-
LLM Perception Limits Affect ARC-AGI Scores
By
–
Good post with maybe big implications for interpreting o3’s ARC-AGI scores: TLDR:
– LLMs bad at ARC because they can’t perceive large text grids
– Solve rates fall as task size in pixels rises, but much later for o3
– 80% of pass@1 o1-mini tasks fail when grids enlarged 2x -
Transformers for NLP and Computer Vision: Comprehensive AI Learning Guide
By
–
Very impressive! >> "Transformers for #NLProc and #ComputerVision: Explore #GenerativeAI and Large Language Models #LLMs with Hugging Face, GPT-4V, DALL-E 3,…" [3rd Edition] http://
amzn.to/3TkBknI
==
Huge, with 728 pages of #AI #DeepLearning #MachineLearning #DataScience joy!