I learned about this trick from this paper "Reversible Vision Transformers" by @Karttikeya_m et al (
https://
arxiv.org/abs/2302.04869). Not an expert here, but I think the reversible transformation is derived from previous work on NICE (
https://
arxiv.org/abs/1410.8516). 6/6
@thegautamkamath
-
Reversible Vision Transformers: Memory-Efficient Deep Learning Architecture
By
–
-
Memory Efficient Neural Networks: Activation Storage Optimization
By
–
This is useful for memory efficient NNs because with careful architecture choice, you don't have to store the activations of every layer. You can store the last layer and work your way backwards. The example indicates that the operation composes, so you can make deep networks 5/n
-
Mathematical Puzzle for Memory-Efficient Neural Networks
By
–
Cute puzzle, courtesy @Karttikeya_m
. Has applications to memory-efficient neural network architectures. Suppose F and G are arbitrary known functions (not necessarily invertible). You are given output O. Can you reconstruct input I? Hint in next tweet. 1/n -
Google Scholar Blog Contributors and AI Research
By
–
I thought that too, but maybe it's like, up to 5. Anurag Acharya is the name I associate with it. But there are a few more people who have written the blog posts here: https://
scholar.googleblog.com -
Google Scholar design control and alternative academic search platforms
By
–
Agreed. The design choices, which are norm setting, are at the whims of a couple of folks (
https://
x.com/thegautamkamat
h/status/1541059471249752066
…). There are alternatives (e.g., @SemanticScholar
). If GS went under, then I assume something else would take its place. -
How Self-Citations Embed Researchers in Academic Literature
By
–
Suppose I write a paper on problem X, and then a year later, I write another paper on problem X. To update the related work on what's new, I start with things that cited my previous paper (again via GS). So by citing others, you end up embedding yourself in the literature. 2/2
-
Citations Boost Research Discoverability and Scholar Impact
By
–
Some say you should be generous with citations, because they cost you nothing. This is not true. You actually *profit* from citing prior work. I learn about a lot of papers via Google Scholar when I get cited. Besides being good scholarship, citing helps discoverability. 1/2
-
External Auditing Requirements for AI Defense Compliance
By
–
Waterloo has the same type of requirement. I assume it's so that a department can't just make an agreement that the defenses are a total farce, and someone external audits the process.
-
Future Works Should Document AI Drawbacks and Limitations
By
–
It sounds like the responsibility of future works to mention these drawbacks, so that hopefully it becomes well known.
-
Cat Herding: Managing Complex AI Systems and Teams
By
–
Volunteer cat herder doesn't have the same ring to it