I wonder if doing softmax in base 2 instead of base e could be a useful optimization on some fixed point hardware. Prepend a 1 to the fractional part and shift left by the integral part as a close-ish approximation of 2^fixed_point.
MACHINE LEARNING
-
Explaining the Internal Reasoning Mechanism of AI Models
By
–
here’s what actually happens inside:
the model maintains three things: • x = the question
• y = current answer
• z = reasoning state (like internal scratch work) each cycle: update z based on x, y, and previous z. then update y based on the new z. think of solving a Sudoku. -
Small 7M Parameter Model Outperforms Large DeepSeek R1 on Reasoning
By
–
a 7M parameter model just beat DeepSeek R1’s 671 billion parameters on hard reasoning tasks. 45% vs 15.8%. trained in hours. fits in 28MB. runs on a single GPU. the trick is that it thinks twice. LLMs generate answers in one pass. make a mistake early, the whole thing
-
AI for Science Social Event at NeurIPS2025 with Leading Researchers
By
–
I’m co-organizing an “AI for Science: Algorithms to Atoms” social event during #NeurIPS2025 with Yann LeCun, Anima Anandkumar, Bill Dally, and Max Welling If you want to talk about AI Scientist, World Models and the future of AI-driven discovery, please come on Dec 5 3:30pm PT!
-
Model outputs text and images with visible CoT
By
–
There’s text CoT (visible in the AI Studio UI) but my understanding is it’s from a single model that outputs both text and images
-
Keras 3 JAX Backend: Easy Adoption with KerasHub Integration
By
–
If you want an easy way to adopt JAX, use Keras 3 with the JAX backend. Need pretrained models? KerasHub gives you access to any model from Hugging Face. All running with native JAX performance.
-
JAX: A Competitive Advantage You Can’t Ignore
By
–
If you're not using JAX, you're missing out on a huge competitive advantage.
-
Fundamental Limits of LLMs at Scale: Mathematical Foundations
By
–
10. Fundamental Limits of LLMs at Scale Establishes mathematical foundations for theoretical limitations constraining LLMs: hallucination, context compression, reasoning degradation, retrieval fragility, and multimodal misalignment.
-
LAMP: Language-Augmented Multi-Agent Reinforcement Learning Pipeline
By
–
9. LAMP: Language-Augmented Multi-Agent RL LAMP integrates natural language processing into multi-agent reinforcement learning through a three-stage pipeline.