Open weights supporter?
Care about transparency?
Sign this asap.
@soumithchintala
-
Open Weights Supporters Called to Sign Transparency Initiative
By
–
-
US Commerce Department Decision on Open Model Weights in 270 Days
By
–
In 270 days, the Department of Commerce will determine whether they will allow open-weights or not. if you support open model weights and want something actionable to do, then figure out how to lobby your opinion to them.
-
Securing Maritime Equipment: Piracy Threats in International Waters
By
–
uh, for starters how would you guard the equipment from pirates? seems like a painted target in international waters.
-
GPT-4 Stochasticity: Variability in Consecutive Model Outputs
By
–
GPT4 seems to have stochasticity — it also gave me 250k (wrong) and 25k (correct) in subsequent tries:
-
H100 GPU Reporting Requirements: AI Models Comparison
By
–
I asked GPT4, Claude and Llama2 to tell me how many H100 GPUs I need before I need to start reporting to the US Government.
They all got it wrong * GPT said 25m GPUs
* Claude said 250k GPUs
* Llama2 gave the gpu-poor answer and said 257 GPUs. The correct answer is 25k GPUs. -
LLaMa2-70B Training Computation Corrected Calculations
By
–
Corrected Calculations (the original claim doesn't change, as it goes from 5e+24 -> 1.1e+24 and is still two orders of magnitude behind):
LLaMa2-70B was trained on 2T tokens, and 1.7m hours of A100 GPU time. At a HFU of ~60%, LLama2-70B took ~1.1e+24 flops (1.7m * 312TFlops -
Multiple Companies Building High-Performance AI Compute Clusters
By
–
more than two companies have clusters with those FLOPs. quite a handful.
-
AI Regulation Unclear on Fine-tuning Reporting Requirements
By
–
The AI Regulation ill-specifies fine-tuning. In this ill-defined state, its not clear if you train a model for 1e25 flops, release it, and someone else trains it further for 1e25 flops, if any reporting has to occur.
-
AI Regulation Loophole: Bitwise Operations Escape Rules
By
–
The AI Regulation says nothing about Bitwise operations — only floating or integer.
Bitwise aficionados go party for now. -
AI Model Size Regulation and Training Compute Requirements
By
–
Regulation starts at roughly two orders of magnitude larger than a ~70B Transformer trained on 2T tokens — which is ~5e24. Note: increasing the size of the dataset OR the size of the transformer increases training flops. The (rumored) size of GPT-4 is regulated.