AI - AI Dynamics

→ View original post on X — @andrewyng

AI community shares diverse visions for the future

By

@andrewyng

–

11 January 2023 20h32

1/2 Thanks everyone that replied to this. Lots of great ideas here! The diversity of our hopes for AI is a sign of how large our field is — we collectively want AI to go lots of places.

Neural Networks: Zero to Hero Series – Building Networks from Scratch

By

–

11 January 2023 20h19

(This will be part of my ongoing series Neural Networks: Zero to Hero https://
karpathy.ai/zero-to-hero.h
tml
… , on building neural networks, from scratch, in code. I have tweeted some of these videos individually already)

→ View original post on X — @steve_rosenbush

Nation-State Cyber Attacks Becoming Commodified Threats

By

@steve_rosenbush

–

11 January 2023 20h06

“A lot of the attacks that used to be reserved for nation-state actors, the sophistication, are starting to be almost commodified,” @Cloudflare CEO Matthew Prince says #CIO #ransomware #cybersecurity #CISO @eastdakota @bellelin_

→ View original post on X — @id_aa_carmack

Nvidia DGX Station GPU Failure and Form Factor Discontinuation

By

@id_aa_carmack

–

11 January 2023 20h06

My $250k DGX station is being replaced for a second time after a GPU failed (first time was the cooling system). I now see that Nvidia has completely exited the form factor.

→ View original post on X — @id_aa_carmack

Regret over NVLink choice versus conventional PCI A100 setup

By

@id_aa_carmack

–

11 January 2023 20h06

In hindsight, I would have been much better off with a more conventional system with PCI A100 cards — I still haven’t done anything dramatic with the extra NVLink bandwidth, and it could have had twice the GPUs.

→ View original post on X — @nvidia

NVIDIA Recognized Again as Best U.S. Workplace by Glassdoor

By

@nvidia

–

11 January 2023 20h05

Thanks to our employees, NVIDIA is again one of the Best U.S. Workplaces, according to @Glassdoor
. Learn more about us: https://
nvda.ws/3CGh72V #GlassdoorBPTW #NVIDIAlife

GPT-2 Pre-training: Hardware Requirements and Token Processing Estimates

By

–

11 January 2023 20h04

Rough example, a decent GPT-2 (124M) pre-training reproduction would be 1 node of 8x A100 40GB for 32 hours, processing 8 GPU * 16 batch size * 1024 block size * 500K iters = ~65B tokens. I suspect this wall clock can still be improved ~2-3X+ without getting too exotic.

Future Plans for GPT-2 Implementation and Educational Content

By

–

11 January 2023 20h04

I'd like to continue to make it faster, reproduce the other GPT-2 models, then scale up pre-training to bigger models/datasets, then improve the docs for finetuning (the practical use case). Also working on video lecture where I will build it from scratch, hoping out in ~2 weeks.

nanoGPT: Simplest Repository for Training Medium-Sized GPTs

By

–

11 January 2023 20h04

Didn't tweet nanoGPT yet (quietly getting it to good shape) but it's trending on HN so here it is 🙂 : https://
github.com/karpathy/nanoG
PT
…
Aspires to be simplest, fastest repo for training/finetuning medium-sized GPTs. So far confirmed it reproduced GPT-2 (124M). 2 simple files of ~300 lines