when doing inference on lots of samples, how much speed up can you expect from increasing the batch size? context: i'm doing inference for lots of images using resnet100 on an a6000 gpu. increased batch size 32 -> 2048 (64x!) and only getting a 20% speedup. how is this possible?
HARDWARE
-
Bounded Search Space Limits Competitive Moat in AI Hardware
By
–
the models are homogeneous, the hardware SKUs are homogeneous, so the overall perf tuning/search space is relatively small. So, the bounded search makes it less moaty — competitors are also searching within the same bounded space.
I cover more on this in my upcoming Latent Space -
REM Detection Device Enables Lucid Dream Control
By
–
Basically it detects REM and triggers these helps you know when you are lucid dreaming, so you would be aware/be able to control your dreams
-
Brain-Computer Interface Cap Enables Direct Neural Robot Communication
By
–
At the bleeding edge of neuroscience, an electronic cap that reads brain waves enables humans to interact with robots using direct, neural communication.
-
AI Breakthroughs: Lucid Dreams, Model Upgrades, New Tools
By
–
Top stories in AI today: -New device aims to induce lucid dreams on demand
-OpenAI rolls out new model upgrades, cost cuts
-Access Copilot's GPTs & plugins on mobile
-Pika releases RPG for prompt comprehension
-6 new AI tools & 4 new AI jobs Read more: http://
therundown.ai/p/lucid-dreami
ng-with-ai
… -
Prophetic Introduces Morpheus-1 Ultrasonic AI Dream Controller
By
–
Prophetic just introduced Morpheus-1, the world's first ‘multimodal generative ultrasonic transformer’.
— Rowan Cheung (@rowancheung) 26 janvier 2024
Designed to explore human consciousness through controlling lucid dreams, the AI device tracks sleep phases and collects dream data to refine AI model.https://t.co/UAPruhPYQ7Prophetic just introduced Morpheus-1, the world's first ‘multimodal generative ultrasonic transformer’. Designed to explore human consciousness through controlling lucid dreams, the AI device tracks sleep phases and collects dream data to refine AI model.
-
Morpheus-1 Device and Major AI Developments from Tech Leaders
By
–
AI NEWS: A new device called Morpheus-1 was revealed to help humans control lucid dreaming. Plus, huge developments from Meta, KitKat, OpenAI, and 6 new tools. Here's everything you need to know:
-
Intel’s Data Center Struggles Amid AI Market Shift
By
–
Intel’s streak ends: Is data center being hurt by the AI craze? Intel’s three-quarter streak of better-than-expected revenue forecasts fell apart as the outlook this evening came in eleven percent lower than expected. Intel’s data center chip sales fell again, capping a
-
AI Accelerators Transform Computing Beyond GPU Architecture
By
–
“These AI accelerators are the new unit of compute. They won’t look like GPUs at all.” Specifically mentions high bandwidth memory, proprietary data, reasoning engines on top of infra, vector data, scaler data. –
@satyanadella -
Non-invasive Brain Machine Interface advances and adoption concerns
By
–
I think you are right. If the first thought of Brain Machine Interface is drilling into the cranium then I get it, I might want to wait on that one myself. But there are so many non obtrusive BMI efforts underway too.