E.g. ~20 years ago Bengio et al 2003 (pdf: https://
jmlr.org/papers/volume3
/bengio03a/bengio03a.pdf
…) trained a neural language model. The state of the art GPT+friends of today are the exact same (autoregressive) model, except the neural net architecture is upgraded from an MLP to a Transformer.
INNOVATION
-
Neural Language Models: 20 Years of Autoregressive Architecture Evolution
By
–
-
Neural Language Models: From Overlooked Niche to AI Breakthrough
By
–
An interesting historical note is that neural language models have actually been around for a very long time but noone really cared anywhere near today's extent. LMs were thought of as specific applications, not as mainline research unlocking new general AI paths and capabilities
-
Perfect commits and feature building best practices guide
By
–
I talk about that a fair bit in these posts:
– https://
simonwillison.net/2022/Oct/29/th
e-perfect-commit/
…
– https://
simonwillison.net/2022/Jan/12/ho
w-i-build-a-feature/
…
– https://
simonwillison.net/2018/Jul/28/do
cumentation-unit-tests/
… -
RAPIDS Platform Accelerates Data Science and AI Efficiency
By
–
Would you like to increase efficiency, streamline development, & achieve more cost-effective data operations in your #AI solution? Join 12/8 and learn how the free @RAPIDSai platform for accelerated data science can help. https://
nvda.ws/3tXGYip #NVIDIAInception #Startups -
Tesla’s Superior Software-Intensive Car Manufacturing Capabilities
By
–
Tesla knows how to make software-intensive cars better than anyone in the world, for one.
-
HyperWrite Automates AI Investor Email Responses
By
–
If you’re in the AI space, you’re probably being bombarded with investor outreach emails.
— Matt Shumer (@mattshumer_) 17 novembre 2022
Fear no more. HyperWrite will respond for you! pic.twitter.com/PlYKxSKSI9If you’re in the AI space, you’re probably being bombarded with investor outreach emails. Fear no more. HyperWrite will respond for you!
-
AI-Powered Ultrasound Innovation Enhances Clinical Collaboration
By
–
With 3.6 Billion #diagnostic exams a year @WHO opportunities to support #doctors & #patients in performance & #care is key. Superb #innovation in #ultrasound here @GEHealthcare @insightdottech https://
insight.tech/health/virtual
-collaboration-in-ultrasound-advances-clinical-care?utm_source=twitter&utm_medium=organic&utm_campaign=2022-tdc-eaves
… #Health #IoT #IntelPartner #ITInfluencer #SDGS #AI #data -
Important Research Beyond Simple Model Scaling
By
–
This blog post explains an important research that does not involve more scaling
-
Kids Empowered Building Satellites Through STEM Education
By
–
Let's democratize capacity to create! Great example for #kids empowering ability to build #satellites Well done! @stoneandchalk @robinsonsystems https://
bit.ly/KidsAeroSpace #Innovation #Aerospace #Space #STEM #STEAM @Shi4Tech #womeninSTEM @GuidoKerkhof @MMach7 @SDGS4GOOD #AI -
AlphaZero Internal Representations Analyzed with Chess Champion
By
–
It was a real honour to closely collaborate with ex-World Chess Champion Vladimir Kramnik on this work, which I think is one of the most thorough analyses yet of the representations and knowledge a complex neural network system like #AlphaZero builds up. Published in PNAS today.