Once we had these datapoints, we needed a way to evaluate answers For this, we relied on LLM assisted evaluation Although this isn't perfect, we think its the best thing out there and are bullish on this in the long run
@hwchase17
-
Custom Agent with Python REPL and Retriever Tools
By
–
Then we could go about improving it! Our final solution was a custom agent with two tools: – Python REPL (for doing analysis)
– Retriever (for dealing with text data present) We're excited to see other's solutions for this as well! -
LangChain Benchmarking Question-Answering CSV Data Tasks
By
–
Recap of important links: Blog: https://
blog.langchain.dev/benchmarking-q
uestion-answering-over-csv-data/
… YouTube: https://
youtube.com/watch?v=jGnf4O
hptbA
… Code & data used: https://
github.com/langchain-ai/l
angchain-benchmarks
… We had a lot of fun doing this and learned a lot – we're going to do it for more tasks! Up next: SQL -
LangChain Benchmarking Question Answering CSV Data
By
–
Blog: https://
blog.langchain.dev/benchmarking-q
uestion-answering-over-csv-data/
… YouTube: https://
youtube.com/watch?v=jGnf4O
hptbA
… Code & data used: https://
github.com/langchain-ai/l
angchain-benchmarks
… Now for a quick thread: -
Building AI Evaluation Datasets from Real User Questions
By
–
The first issue to solve was that we didn't have a dataset to evaluate Rather than try to make up some questions, we put out an example application and logged what questions people asked They also kindly provided feedback, so we could easily identify errors!
-
Improving AI Agents for CSV Question-Answering Applications
By
–
This started ~2 weeks ago, when I tweeted that we wanted to improve our chains/agents for doing question/answering over CSV data Why? Most QA applications focus on text data, but lots of real world data is in CSVs
-
Benchmarking Question Answering Over CSV Data
By
–
Benchmarking Question/Answering Over CSV Data Deep dive on improving an application that does question answering over CSV data: 3000 word blog post
30min video
Open sourced eval data
Open sourced code for gathering feedback
Open sourced final agent code -
Agents with Multiple Retrievers for Conversational QA
By
–
this is pretty easily doable – you can give an agent access to multiple retrievers! https://
python.langchain.com/docs/use_cases
/question_answering/how_to/conversational_retrieval_agents
… -
ParentDocumentRetriever balances chunk size for LLM optimization
By
–
We introduced the ParentDocumentRetriever last week to strike a balance between: – using small chunks during indexing
– passing larger chunks to the LLM Great overview and diagram by @clusteredbytes -
Apps Embracing Open Source Models for Better Integration
By
–
Love to see more apps adding support for open source models!