Even domestically-trained LLMs can be prompt-engineered to generate text that crosses the line.
@hardmaru
-
Model card withheld from external publication
By
–
Model card in the Appendix: “withheld from external publication”
-
PaLM 2 Technical Report Missing Information Disclosure
By
–
They didn’t even disclose this info in “PaLM 2 Technical Report” https://
arxiv.org/abs/2305.10403 -
Importance of Disclosing Model Size and Training Data
By
–
In this particular case, I think it’s perfect fine to tell people how large a model is, and what size is the training data it was trained on. It’s the absolute minimum information one should convey (say, in a model card) anyways, to communicate about the ML model to its audience.
-
ML Trade Secrets Impossible to Keep Due to Research Culture Sharing
By
–
As I’d said before, it is impossible to keep trade secrets in ML, especially at big companies, due to ML research culture. Researchers from all the competing labs all hang out with each other and share all sorts of info about their work, as hackers would. https://
x.com/hardmaru/statu
s/1620318664997339137
… -
PaLM 2 Leak: Parameters and Tokens Revealed, But Key Details Missing
By
–
There’s much talk about Google’s PaLM 2 details being leaked. But seems the only thing “leaked” is unsurprising info: the model has 360B params trained on 3.6T tokens, rather than important details like dataset and architectural tricks. What’s the big fuss?
https://
cnbc.com/2023/05/16/goo
gles-palm-2-uses-nearly-five-times-more-text-data-than-predecessor.html
… -
Tiny Language Models Generate Coherent Text With Simple Vocabulary
By
–
TinyStories: How Small Can Language Models Be and Still Speak Coherent English? A tiny 10M-param transformer can generate paragraphs of coherent text and reason, when trained on synthetic stories limited to only words that a 3-4 year-old would understand.
-
Altman has become the villain of the community according to critics
By
–
"Altman really has become the villain of the community"
https://redd.it/13jhduh -
Sam Altman advocates for US AI regulation and industry permits
By
–
Sam Altman: CEO of OpenAI calls for the US to regulate artificial intelligence He gave several suggestions for how a new agency in the US could regulate the industry—including “giving out and taking away permits for AI companies” Discussion on r/ML: https://
reddit.com/r/MachineLearn
ing/comments/13jhduh/n_sam_altman_ceo_of_openai_calls_for_us_to/
… -
DeepMind’s Robot Based on Open-Platform Robotis OP3 Humanoid
By
–
DeepMind’s robot is based on the off-the-shelf Robotis OP3, and open-platform humanoid project. https://
emanual.robotis.com/docs/en/platfo
rm/op3/introduction/
… Reference: https://
arxiv.org/abs/2304.13653