Auto-Regressive LLMs have a role to play: turning abstract ideas into token sequences (words, actions, code…).
But abstract ideas should be elaborated through planning/reasoning in representation space.
AR-LLMs go directly from prompt to answer, skipping the step of reasoning
LLMS
-
Auto-Regressive LLMs: From Prompt to Answer Without Reasoning
By
–
-
Fine-Tuning Large Language Models on Investment Tweets
By
–
How to Fine-Tune your LLM on Tweets! (large language models for investing)
-
Superalignment team achieves GPT-3.5-level performance with GPT-2 supervision
By
–
Exciting progress from the Superalignment team:
“When we supervise GPT-4 with a GPT-2-level model using this method on NLP tasks, the resulting model typically performs somewhere between GPT-3 and GPT-3.5.” -
Iterative Translation Optimization Using BLEU Score Maximization
By
–
i was essentially doing translation 1000 times, computing the BLEU scores of the translations, and returning the max BLEU score …
-
RAG Solutions for PDF Token Limit Management
By
–
That simply means all the words combined in PDF has exceeded the token limit. The best way is to use RAG (Conversation Retrieval Chain/Agent) to solve the issue. There are templates in our marketplace section that you can refer to
-
AI Chatbots’ Political Persuasiveness: Stanford Research Findings
By
–
Politics are changing with chatbots that “talk” in human-like ways: A team from @StanfordPACS
’s Polarization and Social Change Lab and Stanford HAI explored the boundaries of AI’s political persuasiveness. -
Reading Three Key Papers on Open-Source LLMs and Attention
By
–
oh man, people are focusing on "which paper" I read. I actually read three papers yesterday:
1. Are Open-Source LLM's catching up by @HailinChen3 et. al.: https://
arxiv.org/abs/2311.16989
2. System 2 Attention by @jaseweston and @tesatory : https://
arxiv.org/abs/2311.11829
3. (Part-way through -
Reading Three AI Papers: Open-Source LLMs and System 2 Attention
By
–
oh man, people are focusing on "which paper" I read. I actually read three papers yesterday:
1. Are Open-Source LLM's catching up by @HailinChen3 et. al.: https://
arxiv.org/abs/2311.16989
2. System 2 Attention by @jaseweston and @tesatory : https://
arxiv.org/abs/2311.11829
3. (Part-way through -
Weak Supervision Enables Large Models to Match Human-Level Performance
By
–
Large pretrained models have excellent raw capabilities—but can we elicit these fully with only weak supervision? GPT-4 supervised by ~GPT-2 recovers performance close to GPT-3.5 supervised by humans—generalizing to solve even hard problems where the weak supervisor failed!
-
Weak-to-Strong Generalization: Beyond RLHF for Superalignment
By
–
Naive weak supervision isn't enough—current techniques, like RLHF, won't be sufficient for future superhuman models. But we also show that it's feasible to drastically improve weak-to-strong generalization—making iterative empirical progress on a core challenge of superalignment