AI pioneer Geoff Hinton says the future of computing is analog, something I’ve long believed. I think he’s right. And, the term mortal computation is a great moniker. // @geoffreyhinton // #computing #AI #deeplearning #mortalcomputation // @NeurIPSConf
RESEARCH
-
Initial Fine-Tuning Approach: Breaking Books into Chunks and Generating Questions
By
–
initial approach: oh this seems not too bad, I read this doc https://
beta.openai.com/docs/guides/fi
ne-tuning
… and was like yeah I can just break up the authors' books into chunks and generate some simple questions for each chunk and then use that to fine-tune the model -
Fine-tuning models is easy but data preparation remains challenging
By
–
it was easy to fine-tune the models (sorta, gathering and cleaning the data sucked and it doesn't look like there is any easy automatic process to do this yet in terms of a simple online tool)
-
Animashree Anandkumar Featured in AAAS Member Spotlight
By
–
Thank you @aaas https://
aaas.org/membership/mem
ber-spotlight/animashree-anandkumar-uses-ai-connect-scientific-fields
… -
Stanford and Google Improve Transformer Training with Convex Analysis
By
–
Stanford U & Google’s Convex Analytic Training Framework Improves the Understanding and Optimization of Transformers https://
syncedreview.com/2022/12/01/sta
nford-u-googles-convex-analytic-training-framework-improves-the-understanding-and-optimization-of-transformers/
… -
MIT Scientists Use GPT-3 LLM for Clinical Natural Language Processing
By
–
Scientists at @MIT_CSAIL have used a GPT-3-style large language model (LLM) to perform tasks such as drug regimen extraction – an approach that will transform clinical natural language processing. Code > https://
bit.ly/3OK3CEd via @antgrasso #NLP #gpt3 #AI -
Acknowledgment of researchers for their collaborative work
By
–
Thanks to researchers @michiyasunaga, @ArmenAgha
, @WeijiaShi2
, @jure
, @percyliang
, @ml_perception
, @LukeZettlemoyer
, @scottyih and the rest of the team for all their work on this! 4/4 -
RA-CM3 Scalable Modular Knowledge Integration Architecture
By
–
RA-CM3 integrates knowledge in a more scalable & modular way compared to previous methods. It’s more resilient to long-tail or unseen knowledge which can help the growth & update of knowledge over time in real world settings. We’re excited for the possibilities it presents. 3/4
-
RA-CM3 Model Outperforms Baselines with 70% Less Compute
By
–
The RA-CM3 model significantly outperforms baseline multimodal models on both image & caption generation tasks, while using <30% of the compute of comparable models & exhibiting novel capabilities like knowledge-intensive image generation & multimodal in-context learning. 2/4
-
Meta AI Introduces RA-CM3: Multimodal Model for Text and Image Generation
By
–
New paper from our team at Meta AI. Retrieval-Augmented CM3 (RA-CM3) is the first multimodal model that can retrieve and generate mixtures of text and images — while also reducing training cost and model size. Now available on arXiv https://
arxiv.org/abs/2211.12561 1/4