In the last sentence of his reply he says "in a world where" people other than OpenAI can train an LM, this watermarking fails. I guess he doesn't realise we're in that world right now, when it comes to restating/styling/summarising prose.
@jeremyphoward
-
Fine-tuned Language Models for Text Rephrasing Projects
By
–
I'm sure we'll see lots more projects pop up that let you "rephrase" an input text in all kinds of ways using fine-tuned language models. Many of them will be open source.
-
LLM Detection Methods Face Fundamental Technical Challenges
By
–
Am I missing something, or are all these attempts at recognising LLM outputs obviously destined to fail?
— Jeremy Howard (@jeremyphoward) 5 janvier 2023
It's dramatically easier to train an LLM for rewording a text than creating the text in the first place; then add a loss func that incorporates detection avoidance. https://t.co/TO0zIGHoAeAm I missing something, or are all these attempts at recognising LLM outputs obviously destined to fail? It's dramatically easier to train an LLM for rewording a text than creating the text in the first place; then add a loss func that incorporates detection avoidance.
-
AI Watermark Detection Bypass Using Paraphrasing Tools
By
–
There's already an easy workaround. The "watermark" will surely disappear if you run your text through @TheQuillBot or @wordtune etc. (And those tools weren't even built for this purpose!)
-
Curriculum Learning and Dropout Improve Model Performance
By
–
Thanks to using a form of curriculum learning (5 epochs) and better dropout (20 & 50 epochs) I've now been knocked off the leaderboard entirely! Here's the latest:
-
Model Evaluation Trade-offs: Epochs versus Second-Order Methods
By
–
Sure. But you only get to do 60,000*20 model evals in total either way! (i.e if the 2nd order method needs twice as many evals, you only get 10 epochs)
-
DDPM Course Implementation From Scratch Explained
By
–
We've done ddpm from scratch in the latest course. Lemme know if you want more info
-
Dropout Improves Model Performance to 95.2% Accuracy
By
–
After adding dropout, I increased the 5 epoch best to 93.2%, and christopherthomas on http://
forums.fast.ai improved the 50 epoch best to 95.2% – first time we've gotten a result >95%! 😀 -
Training ResNets Without Batchnorm: Achieving Strong Results
By
–
Pretty amazing that we can get such good results even without batchnorm — and not using a proper resnet init like Fixup either! Maybe I should try lsuv with batchnorm-free resnet sometime…
-
Bug Fix Improves Model Accuracy to 94.9 Percent
By
–
h/t to @PiotrCzapla for noticing I had a bug that meant there's no batchnorm in those models! After fixing, my new best results: Epochs / c
5 / 93.0
20 / 94.5
50 / 94.9