Anthropic Messages Use Anthropic’s new beta Messages API and Claude 2.1 with the new
@langchain x @AnthropicAI packages in Python and JS! With better support for processing your Chat Messages in the correct format, give ChatAnthropicMessages a go!
@langchain
-
Anthropic Messages API and Claude 2.1 Integration with LangChain
By
–
-
Testing LLM Agents: Benchmarking Guide and Performance Evaluation
By
–
11/ Test your own agent Want to test out a new LLM? Agent architecture? Prompting strategy? Run these benchmarks on your own agent by following this easy guide: https://
langchain-ai.github.io/langchain-benc
hmarks/notebooks/tool_usage/intro.html
… Performance is a function of the whole system. We were only able to clearly run a small -
Benchmarking Agent Tool Use: Error Analysis and Results
By
–
12/ Final Note on the Charts Error bars computed as standard error, which is why the smaller datasets have wider error bars. Learn more at the blog post! Learn more at the blog post! https://
blog.langchain.dev/benchmarking-a
gent-tool-use/
… Special thanks to @veryboldbagel and @WHinthorn for putting -
Platform Reliability and Training Data Alignment in AI Systems
By
–
10/ Observations – Bigger isn't always better: it all depends on the alignment of task training data – The big platforms are still flaky! Lots of internal server errors we had to work around. We left out some platforms because they were too unreliable still!
-
Typewriter Task: Agent with 26 Letter Tools
By
–
4/ Task 2: Typewriter (26 tools) Agent has 26 tools, 1 for each English letter. It has to type the provided word. https://
langchain-ai.github.io/langchain-benc
hmarks/notebooks/tool_usage/typewriter_26.html
… -
LangChain Typewriter Task: Agent Tool Usage Benchmark
By
–
2/ Task 1: Typewriter (1 tool) Agent has 1 tool (a typewriter). It has to type the provided word. https://
langchain-ai.github.io/langchain-benc
hmarks/notebooks/tool_usage/typewriter_1.html
… -
GPT-4 Typewriter Task Performance Evaluation and Limitations
By
–
3/ Typewriter results None of the agents are perfect. GPT-4 had a hard time typing "keyboard" and "head" https://
smith.langchain.com/public/ff14ecb
2-3664-4c4a-b2dc-d8aa9fd2185d/d
… -
New Open-Source Tool Use Benchmarks for LLM Agents
By
–
Agents are the “killer” LLM app, but building and evaluating agents is hard. A huge part of agents is tool use, but there aren't enough open-source tool use benchmarks out there. Today, we are excited to release four new test environments for benchmarking LLMs’ ability to
-
LangChain Redis RAG Template Financial PDF Analysis
By
–
LangChain & Redis over Financial PDFs Excited to work with the @Redisinc team to create a RAG template for developers to chat with public financial PDF documents such as Nike’s 10k filings. The application uses: FastAPI and Uvicorn to serve client requests via HTTP
-
Personalizing Videos with LLMs Using LangChain
By
–
Personalizing videos with LLMs The talented folks @RubricLabs + @withgraphite wrote up a deep dive on using LangChain to generate personalized year-end "GitHub wrapped" videos for everyone! Generate your own: http://
year-in-code.com Blog: