It is very much worth the subscription cost! I am good at getting information with Google, but ChatGPT is often better, and can be more thought provoking.
@id_aa_carmack
-
Architectural Cleverness Cannot Compensate for Raw Scaling Gap
By
–
http://
incompleteideas.net/IncIdeas/Bitte
rLesson.html
… is about the surprisingly limited impact of architectural cleverness versus raw scaling. I am analogizing to suggest that a radical hardware architecture like https://
en.m.wikipedia.org/wiki/Cerebras probably won’t make up for being back a few stages, but hope springs -
China’s Fab Capabilities and Cerebras Wafer-Scale Integration Viability
By
–
China’s fabs are several generations behind, but I wonder if something like Cerebras’ wafer scale integration would still be viable. The limited impact of clever architecture vs simple scaling is one of the “bitter lessons”, but it is still meaningful.
-
PyTorch 2.0: torch.set_default_device() Simplifies Device Management
By
–
PyTorch 2.0 brings torch.set_default_device(dev), which cleans up tons of device=dev and to(dev) code fragments. It also allows models to be constructed directly on device, which makes the random init of big models much faster.
-
UK as AI Innovation Haven: DeepMind Pride and Regulatory Tensions
By
–
I would cheer for the UK staking out a position as a haven for AI innovation. DeepMind should already be a source of national pride, but there would be some tension, because DeepMind/Google is at the stage of actually favoring some regulation that cements their position.
-
Hand-written optimization libraries achieve near-optimal performance across architectures
By
–
For a specific task, like doing a large matrix multiply, there are libraries that get nearly optimal performance across different CPU and GPU architectures, but only because programmers have gone and hand written each case. Generally, software does “just work” across lots of
-
Optimizing Deep Networks for Datacenter Performance: Complexity and Cost Tradeoffs
By
–
It is harder than you think! It is a fun exercise to consider optimizing deep nets for high cycle times across a datacenter, but it is not at all straightforward, and the “cost of computation” goes way up versus maximizing the work that a single system can do.
-
AGI Speed Advantage May Not Be As Obvious As Assumed
By
–
The notion that AGIs will “think 10,000 times faster than us” isn’t as obviously true as it seems at first glance. It is under appreciated that the self driving teams struggle to make their systems operate at barely real time rates, and they are not close to AGI systems. Step
-
Unexpected Success of Midjourney: Updated Predictions on AI Tools
By
–
I would not have bet against ChatGPT user counts, but I probably would have lost a bet on the scale of @midjourney financial success if @DavidSHolz had caught me in an argumentative mood (not sure he would have predicted it either!) — I have updated my priors.
-
AI Will Change the World but More Slowly Than Expected
By
–
My big bets are with my time, and I do bet on AI changing the world, but it will take longer than the more enthusiastic people predict.