Lots of folks have asked how we achieved 49% on SWE-bench Verified with the new Claude 3.5 Sonnet, beating the previous SOTA of 45%. Here's how we did it:
@alexalbert__
-
Claude Now Available as Model Option at Same Price
By
–
Yep same price, no catch. Claude is just now available as one of the model options
-
GitHub Copilot News from Anthropic
By
–
Read more here: http://
anthropic.com/news/github-co
pilot
… -
Claude 3.5 Sonnet Now Available on GitHub Copilot
By
–
Excited to announce that Claude is now available on GitHub Copilot. Starting today, developers can select Claude 3.5 Sonnet in VS Code and GitHub. Access will roll out to all Copilot Chat users and organizations over the coming weeks.
-
Anthropic Recruiting Prompt Engineers with Language Model Experience
By
–
p.s. if you've also spent a borderline unhealthy amount of time talking to Claude and have built cool projects that use language models, you might be a great fit for our team take a minute to apply to our prompt engineering role here: https://
jobs.lever.co/Anthropic/e3cd
e481-d446-460f-b576-93cab67bd1ed
… -
Prompt Engineering Team Announces Upcoming Exciting Developments
By
–
stay tuned for more from the prompt engineering team very soon, we have lots of exciting stuff in the pipeline
-
Tips and Tricks from Chatting with Claude at Anthropic
By
–
in the past couple months since I joined Anthropic, I've spent a lot of time talking to Claude
— Alex Albert (@alexalbert__) 28 août 2023
from all those chats, I've picked up on a few tips and tricks that I wanted to share… https://t.co/he57MLQi5rin the past couple months since I joined Anthropic, I've spent a lot of time talking to Claude from all those chats, I've picked up on a few tips and tricks that I wanted to share…
-
Alex Albert Joins Anthropic as Resident Prompt Engineer
By
–
back from my twitter hiatus with some personal news… I'm excited to share that I've joined @AnthropicAI as a resident prompt engineer!
-
Complex Tasks Cause Knowledge Forgetting During Execution
By
–
I think the interesting observation here is that in the process of completing a larger, more complex task, it seems to "forget" this knowledge
-
Word Length Memorization Works for Common Words
By
–
yeah I understand that and you're right it might fail on edge cases, but for most common words (you can test all the words it produced in the output I posted to see for yourself) it has indeed "memorized" the right answer for the length of the word