The information bandwidth of the human visual system is *not* 20MB/s just because you have 1 million optic nerve fibers. It's exactly like saying that the information bandwidth of a 1280×720 video is 16MB/s, because you have 1280x720x3 channels with 256 possible values firing at
@fchollet
-
Best theories on hippocampal and cortical column algorithms
By
–
Who has the best theories describing the algorithms / data structures implemented by the hippocampal system and / or cortical columns?
-
Criticism of intelligence measurement understanding
By
–
Someone hasn't read "On the measure of intelligence"…
-
Simple Multi-Backend Model for Sequence Similarity Estimation
By
–
Here's a simple model I wrote yesterday (it learns to estimate a similarity metric between pairs of sequences). It runs with all backends — no code changes. Tried PT -> trains at 24ms/step on V100. Tried JAX -> trains at 10ms/step on V100.
-
Keras and PyTorch: Prototype Once, Deploy Anywhere with JAX
By
–
I know some companies like to prototype in PyTorch then rewrite in JAX for performance. Did you know you could just… prototype in Keras + PyTorch, then switch to JAX *while keeping all of your model code*? At most you'll have to rewrite the train_step or the training loop.
-
Essential Keras Tips for Debugging and Prototyping with Different Backends
By
–
Essential Keras tips: 1. Debug/prototype with eager execution. You can prototype with the PyTorch backend, or even with the NumPy backend if you're only looking at a layer or a forward pass (the NumPy doesn't support gradients/training). With JAX or TF, make sure to use
-
Active Inference as a Solution to Distribution Shift in Models
By
–
There are essentially two main options to remedy this: 1. Find ways to perform active inference, so that the model adapts its learned program in contact with a new data distribution at test time. Would likely lead to some meaningful progress, but it isn't the ultimate solution,
-
Neural Networks Struggle with First-Order Logic Learning
By
–
As pointed out by @sirbayes
, this paper https://
starai.cs.ucla.edu/papers/ZhangIJ
CAI23.pdf
… has a formal investigation into the observation from the first tweet — that "my models were completely unable to learn to perform actual first-order logic — despite the fact that this ability was definitely part of -
Generative AI Revenue Market Size and OpenAI’s Market Share
By
–
My tweet says 4B (2B consumer + 2B enterprise). OpenAI is half of that. Note that the relevant quantity is gen AI derived revenue from companies that train their own models and directly monetize them. Not all revenue made by anyone using gen AI (e.g. some company using a 3rd
-
AI Chip Economics: VC Loss, Big Tech Profits, Uncertain Returns
By
–
~1/3rd of the chips are bought via venture capital that won't see a return. 2/3rds are big tech. That portion is mostly to provision cloud services (which are very profitable). In turn, these cloud services are sold to companies which may or may not see a return from their models