Is it the number of examples that matters or the number of presentations to the model during training? E.g. humans used spaced repetition to memorize facts but there are no equivalents of similar techniques in LLMs where the typical training regime is uniform random.
LLMS
-
MIT Scientists Develop Liquid AI Model From Brain Dynamics Equation
By
–
MIT scientists solved an equation in brain dynamics and used that to design a new liquid AI model – a potential new building block of future intelligent systems: https://
bit.ly/3EtPf2r Code repository: https://
bit.ly/3VfC63N -
Contrastive Language-Audio Pretraining with Feature Fusion
By
–
Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation Wu et al.: https://
arxiv.org/abs/2211.06687 #Artificialintelligence #DeepLearning #MachineLearning -
OpenAI’s Trajectory Toward Advanced Content Creation Models
By
–
Great writers are prolific readers. GPT-3 read everything on the internet. Arguably, @openai is *on the path* to create the best content creation machine possible.
-
Evaluation Metrics for Language Modeling Explained
By
–
Evaluation Metrics for Language Modeling https://
thegradient.pub/understanding-
evaluation-metrics-for-language-models/
… -
MrsFormer: Multiresolution Attention Cuts Transformer Costs
By
–
‘MrsFormer’ Employs a Nove Multiresolution-Head Attention Mechanism to Cut Transformers’ Compute and Memory Costs https://
syncedreview.com/2022/11/14/mrs
former-employs-a-nove-multiresolution-head-attention-mechanism-to-cut-transformers-compute-and-memory-costs/
… -
LangChain 0.0.13 Release: Vector DB QA and Documentation Updates
By
–
LangChain Version 0.0.13 Question/Answering w/ a vector DB chain (demo coming tmrw) Loading a prompt from a text file (
@edmarferreira first commit!) Misc cleanup (Eugene x4!!!) Big Documentation overhaul (w/ Eugene again) -
CRINGE Loss: Learning What Language Not to Model
By
–
The CRINGE Loss: Learning what language not to model Adolphs et al.: https://
arxiv.org/abs/2211.05826 #ArtificialIntelligence #DeepLearning #MachineLearning -
GPT-2’s Inscrutable Internal Matrices Remain Poorly Understood
By
–
Nobody's ever even going to understand how GPT-2 worked, except that there sure were a lot of inscrutable matrices in there.