In the absence of that, at least aim to "first, do no harm" — which means don't propose an intervention at all unless you're confident you know the first, second, and third order effects over a long time scale across all impacted groups.
@jeremyphoward
-
Compute Governance Risks and International Coordination Challenges
By
–
It seems likely that compute governance and international coordination will make those risks more likely and more dangerous. I appreciate the time you've put into studying risks, but I think you need to put in *much* more time into studying specific interventions and impacts.
-
LoRA vs Full Fine-tuning: Combining Techniques for Model Optimization
By
–
I've been experimenting with that recently. It seems pretty similar. You can do both BTW – LORA for initial layers and full f/t for final
-
Freezing Layers and Discriminative Learning Rates in Fine-Tuning
By
–
Very happy to see more folks supporting what I've been saying for years — it really is a good idea to freeze layers when fine tuning. (And it's also a good idea to use discriminative learning rates.) See our ULMFiT paper for details (which is from 2018, but is still correct.)
-
Internal Capabilities and Functional Components of AI Systems
By
–
You're saying only people have capabilities? A chair has capabilities. A chair inside a box has internal capabilities. You have to open the box to access those capabilities. There is no anthropomorphization going on here — it's a description of the components of a function.
-
Training AI Models Requires More Than Simple Code
By
–
Training a model of this type does not require merely writing 20 lines of code.
-
Internal capabilities in AI models beyond fine tuning explained
By
–
That oft-repeated phrase is technically accurate (if we ignore fine tuning) but doesn't imply anything about what capabilities are developed internally to allow that to happen.
-
Neural Networks Implicit Spatial Maps Discovery
By
–
A lot of folks surprised to discover neural nets create implicit spatial maps, based on a paper that came out yesterday. But as you see here from our 2018 course, this has been known for some time! This example is from a 2016 paper which describes a Kaggle gold medal approach.
-
Exciting AI Collaboration Opportunity with Leading Researcher
By
–
Listen to this bit, where I told the students in the course that I thought this idea had great potential — and that I was particularly excited about the opportunity to team up with @seb_ruder
… -
First Fine-Tuned Universal Language Model Demonstrated 2017
By
–
Just happened across this video: towards the end of 2017 – this clip from the recording of http://
fast.ai lesson 4 was the first time the effectiveness of a fine-tuned universal language model was demonstrated. (Our IMDB dataset result shown at the top.)