yeah it’s similar— I think you could call it multi-token sparsity or something like that? I mean obviously the activations have to be sparse in this case but we can use as many discrete tokens as we want (unlike in the projects you’re mentioning)
MACHINE LEARNING
-
Discrete Tokens vs Continuous: Perplexity Trade-offs
By
–
also in this setup you probably need a much larger number of discrete tokens than continuous ones to get decent ppl
-
Logit Lens Limitations in Early Layer Analysis
By
–
nah that’s not the same thing at all; logit lens is pretty poorly conditioned esp at early layers and certainly doesn’t capture all of the hidden state
-
Hudson River Trading Discusses AI for Price Discovery Quantitative Finance
By
–
Hear Hudson River Trading (@wehrtyou) speak about how they use #AI for price discovery and trading in #quantfinance. Join the session at #GTC24 on March 19. https://t.co/BZ5rOlwc2y pic.twitter.com/VLAsk8n6ym
— NVIDIA AI (@NVIDIAAI) 15 février 2024Hear Hudson River Trading (
@wehrtyou
) speak about how they use #AI for price discovery and trading in #quantfinance. Join the session at #GTC24 on March 19. https://
nvda.ws/42GV6ge -
Latent Text Transformer: Making Model Thoughts Readable
By
–
random research idea: Latent Text Tansformer (LTT) in a nutshell: replace sequence of *vectors* as hidden states of the transformer with sequences of *tokens*, so we can read the model's "thoughts" directly then train a transformer that uses longer sequences of *discrete
-
Key Requirements for IoT Success in Digital Transformation
By
–
#Infographic: Key Requirements for IoT Success
Via @ingliguori #DigitalTransformation #Cloud #MachineLearning #BigData #ArtificialIntelligence #cybersecurity #Blockchain #DX #Analytics #AI #IIoT #DataScience #IoT #IoTPL #iotcommunity #IoTCL #Industry40 #IndustrialIoT #industryIoT -
Post-Op Infection Prevention: CDC Gaps and AI Solutions
By
–
Austin’s Post-Op Infection Highlights Lag In CDC’s Overall Prevention Effort https://
forbes.com/sites/michaelm
illenson/2024/01/19/austins-post-op-infection-highlights-lag-in-cdcs-overall-prevention-effort/?utm_source=dlvr.it&utm_medium=twitter
… #Innovation #ArtificialIntelligence #MachineLearning -
Yamaha Motoroid 2: AI-Powered Motorcycle That Learns and Adapts
By
–
Motorcycle YAMAHA MOTOROiD2 that remembers and adjusts to you, standing upright on its own, thanks to Al technology and high precision sensors laranjasolta v/ @gunsnrosesgirl3 #innovation #AutonomousVehicles #ElectricVehicles #AI #ML @HaroldSinnott @Xbond49 @helene_wpli
-
Comprehensive AI and Data Science Infographic Guide
By
–
Explore the vast universe of AI & Data with this comprehensive infographic! From machine learning to natural language processing, it's a treasure trove for any #DataScientist. For a deep dive into AI, be sure to follow @ingliguori
. #AI #MachineLearning #BigData #Fintech -
Efficient Fine-tuning with Adapters for LLMs
By
–
Want to learn about efficient #finetuning techniques, why #adapters are the future of #LLMs, and how to get started? Save your spot for our upcoming ML Real Talk for an informative and interactive conversation on adapter-based fine-tuning. https://
pbase.ai/3I2x1Y0