Also why no 70B model?
1) Because of the 4x smaller dataset (compared to Llama 2 pretraining)?
2) to make 100k context windows work hardware-wise?
AI HARDWARE
-
Why no 70B model for 100k context windows
By
–
-
Groq offers fastest GPU supply chain and production time
By
–
Are you tired of waiting for GPUs? @GroqInc has a domestic supply chain and the fastest time to production on everything from hardware to getting your models up and running. Reach out to see a demo and have a conversation: contact@groq.com
-
Black Hornet Nano Drone Engineering Innovation Details
By
–
More details: https://
engineering.com/story/the-incr
edible-engineering-of-the-black-hornet-nano-drone
… -
Norway Supplies 1,000 Black Hornet Nano Drones to Ukraine
By
–
These Black Hornet Nano drones are quite something: * 5 inches diameter
* weighs 16 grams * 1 mile range (24mins)
* live hi-def video
* operates silently * cost $195,000 Norway just gave 1,000 to Ukraine -
Groq LPU: AI Hardware Outperforming Graphics Processors
By
–
The Groq Language Processing Unit™ (LPU™) was built for AI from the ground up, enabling Groq to consistently outperform graphics processors on AI and ML tasks. Email contact@groq.com to learn more.
-
Llama 2 Apps and Experiences on MediaTek Devices
By
–
We're excited to see all the new apps and experiences enabled by Llama 2 on MediaTek-powered devices!
-
Hardware Dependencies and Limitations for Metal Support
By
–
Interesting, thanks for sharing. This sounds pretty cool actually. But afaik certain features are also hardware dependent, like bfloat16, so I am not sure if Metal or macOS can address that. Would probably require new chips as well.
-
QLoRA 4-bit NormalFloat format supported only on Nvidia GPUs
By
–
Ah sorry, I meant M1/M2 chips (not specifically M1/2 CPUs). As far as I know, the 4-bit NormalFloat format that is used in QLoRA is currently only supported on Nvidia GPUs (
https://
github.com/TimDettmers/bi
tsandbytes/issues/485
…). Maybe the repo you mentioned uses a different type of quantized training. -
Alternative Precision Flag for Non-GPU Machines
By
–
I haven't tried on non-GPU machines, but maybe the following works:
"–precision 16-true" instead of "–precision bf16-true" -
BFloat16 Support in Ampere GPUs and M1 Chips
By
–
BFloat16 is a format introduced in Ampere architectures (Nvidia A100 cards) or newer. M1 chips for some reason have also bfloat16 in PyTorch, but I think this MPS bfloat16 is something very different.