To improve the faithfulness of model-generated reasoning, we study two other ways of eliciting reasoning from models in Radhakrishnan et al. These methods rely on question decomposition, or breaking down a question into smaller subquestions to help answer the original question.
RESEARCH
-
Testing Chain of Thought Reasoning Faithfulness in AI Models
By
–
We make edits to the model’s chain of thought (CoT) reasoning to test hypotheses about how CoT reasoning may be unfaithful. For example, the model’s final answer should change when we introduce a mistake during CoT generation.
-
Chain of Thought Impact on Model Reasoning Accuracy
By
–
For some tasks, forcing the model to answer with only a truncated version of its chain of thought often causes it to come to a different answer, indicating that the CoT isn’t just a rationalization. The same is true when we introduce mistakes into the CoT.
-
Chain-of-Thought Performance: Computation vs Reasoning
By
–
We check if CoT provides a performance boost because of the greater computation provided by a longer input alone (left) or due to information encoded in the particular phrasing of the reasoning (right). Our work provides evidence against these hypotheses.
-
Measuring and Improving Language Model Reasoning Faithfulness
By
–
When language models “reason out loud,” it’s hard to know if their stated reasoning is faithful to the process the model actually used to make its prediction. In two new papers, we measure and improve the faithfulness of language models’ stated reasoning.
-
Do AIs Understand Logic Puzzles? Reasoning Test
By
–
Understand logic puzzles "Respond to this question only based on the information provided here. Cats like dogs, and dogs like rabbits. Cats like anything that dogs like. I really really dislike rabbits. How do cats feel about rabbits?" https://replicate.com/p/fqch35bbyrcr j7zhl7zxb5k6ha
… -
Code Generation: BERT Training on IMDB with Distributed Data Parallel
By
–
> It can write code. "Write a python script that trains `bert-large` on the `IMDB` dataset using the Transformers `Trainer` class and Datasets library. I have access to four GPUs, so let's use DDP." https://
replicate.com/p/gzcomldbikya
y2s6q6744lsyt4
… -
Facebook Research Llama GitHub Repository Link Shared
By
–
github: https://
github.com/facebookresear
ch/llama
… -
Meta Releases Llama 2 Open Foundation and Fine-Tuned Chat Models
By
–
Meta releases Llama 2: Open Foundation and Fine-Tuned Chat Models paper: https://
ai.meta.com/research/publi
cations/llama-2-open-foundation-and-fine-tuned-chat-models/
…
blog: https://
ai.meta.com/llama/ develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion -
MIT AI Professor Patrick Winston’s Complete 6.034 Lectures
By
–
RIP to MIT professor Patrick Winston, who passed away #otd in 2019. He was director of MIT’s AI lab for 25 years. Here are his complete video lectures for 6.034 (Artificial Intelligence): http://
bit.ly/PHWvideos