Prompt: "You are a GPT and you're in charge of training an even better GPT, congrats! You have a dataset here . You can train it on document chunks like this: and sample its current understanding like this: . And here's a calculator and a scratchpad . Begin:"
AI
-
Growing Up in the Virtual Fifties: Nostalgia and Tech Evolution
By
–
I like to say that we grew up in the (virtual) 50s.
-
Meta-learning policies for LLM attention management during training
By
–
Feels like a lot of fertile ground is left in managing the "attention" of an LLM during its training via a meta-learning policy, instead of the typical "memorize dataset uniformly at random" strategy. And giving it a calculator and a scratch pad.
-
Training Strategies: Skimming, Filtering Noise, and Revisiting Content
By
–
More generally a few remarkable strategies people use during their training:
1) skim text because they already know it
2) ignore text because it's clearly noise (e.g. they won't memorize SHA256 hashes. LLMs will.)
3) revisit parts that are learnable but not yet learned -
Examples vs. Presentations: Spaced Repetition in LLM Training
By
–
Is it the number of examples that matters or the number of presentations to the model during training? E.g. humans used spaced repetition to memorize facts but there are no equivalents of similar techniques in LLMs where the typical training regime is uniform random.
-
Datasette Now Runs in WebAssembly Browser Environment
By
–
… update to this thread: I got Datasette working in WebAssembly! https://
lite.datasette.io and -
Decentralized AI Through Web3 Agents and AI Nodes
By
–
Blockchain.AGI.Eth Web3 AI Agents X AI Nodes = Decentralized AI Preliminary Presentation: https://
montreal.ai/aiagents.pdf #AGI #AINodes #AIAgents #AIToken #MontrealAI -
CR-SQLite WASM Implementation Now Available
By
–
Oh wow that thing's already working in WASM too? Exciting! https://
github.com/vlcn-io/cr-sql
ite/tree/main/pkg/wasm-esm/crsqlite
… -
Stanford HAI Spring Conference 2023: AI, Creativity, Policy
By
–
A sneak preview of the @StanfordHAI Spring Conference 2023 – AI, generative models, arts, creativity, policy, regulation, and more! Stay tuned and sign up to our mailing list: https://
hai.stanford.edu/subscribe-hai-
mailing-list
… -
Datasette Lite adds GitHub JSON loading capability
By
–
Added a new feature to http://
lite.datasette.io – it can now load JSON data directly from GitHub (or other CORS-friendly hosting) using the new ?json= URL parameter Documentation here: https://
github.com/simonw/dataset
te-lite#loading-json-data
…