RLHF sacrifices in-context learning for ease of prompting. Model is more capable in an informal, less quantifiable sense. Some contrived tasks are nearly impossible without tuning though e.g. summarizing text that’s close in length to the size of the context window.
AI
-
AI Models and the Challenge of Original Ideas
By
–
yes the model can design and order diamondoid bacteria but it’s just plagiarizing the training set it will never have original ideas of that caliber there will always be demand for hu
-
Databricks Dolly: Open Source LLM Training in 30 Minutes
By
–
Have 30 minutes to spare? That’s all you’ll need to train your own Dolly – an open source LLM – on Databricks and enjoy #ChatGPT-like capabilities. Check out our latest updates, including our GitHub repo, here https://
databricks.com/blog/2023/03/2
4/hello-dolly-democratizing-magic-chatgpt-open-models.html
… -
Kortschak Scholar Program Recruits Exceptional AI Students at Caltech
By
–
The Kortschak scholar program has helped us recruit exceptional students at @caltech including @ZongyiLiCaltech @guanzhi_wang Thanks to @WKortschak @MooreFound We will miss Gordon Moore.
-
Gary Marcus on Why GPUs Won’t Lead AI Future
By
–
Scientist and AI expert Gary Marcus explains that our world is governed by the chips that we know how to build and GPUs aren't likely to be the chips that advance AI in the future. Read how @GroqInc can address this problem: http://
groq.link/meetgroqcompil
er
… https://
youtu.be/Puo3VkPkNZ4?t=
871
…. -
Chat Interface: The Biggest UI/UX Revolution
By
–
Chat interface to everything is the biggest revolution in UI/UX https://t.co/YKVqoR9q5Y
— Marek Rosa | European🇪🇺 | South African🇿🇦 (@marek_rosa) 25 mars 2023Chat interface to everything is the biggest revolution in UI/UX
-
ChatGPT Can’t Fix Airplane Tires Yet
By
–
Pilot just told us the plane needs a new tire. Everyone (plus bags) must unload. Can’t help but think that ChatGPT could have fixed it with us still onboard.
-
H2OWorld Conference Coming to India for the First Time in April
By
–
For the first time ever, #H2OWorld is coming to #India! After sold-out events in San Francisco, New York, London, Sydney, and other global locations, our conference is set to take place in Bengaluru, on April 19th. Register here: https://
townscript.com/e/h2o-world-in
dia-112201
… #DemocratizeAI -
GPT-4 Report Lacks Reproducibility Documentation
By
–
While we're looking at source codes, I checked out the source of the GPT-4 Tech Report, and found neurips_checklist.tex (not compiled in their main.tex, I added it back in). Reproducibility answers are DEFINITELY not for this paper https://
arxiv.org/abs/2303.08774 -
NAS and Transformer Architecture Search Space Subtleties
By
–
Yes, I totally agree that the fact that the proxy task was Transformer(base) is quite subtle in So et al. (just that one sentence). NAS is generally a once-per-search-space/problem-type (sort of like human researchers coming up with a new model architecture, just automated).