Once we had these datapoints, we needed a way to evaluate answers For this, we relied on LLM assisted evaluation Although this isn't perfect, we think its the best thing out there and are bullish on this in the long run
@hwchase17
-
LangChain Benchmarking Question-Answering CSV Data Tasks
By
–
Recap of important links: Blog: https://
blog.langchain.dev/benchmarking-q
uestion-answering-over-csv-data/
… YouTube: https://
youtube.com/watch?v=jGnf4O
hptbA
… Code & data used: https://
github.com/langchain-ai/l
angchain-benchmarks
… We had a lot of fun doing this and learned a lot – we're going to do it for more tasks! Up next: SQL -
LangChain Benchmarking Question Answering CSV Data
By
–
Blog: https://
blog.langchain.dev/benchmarking-q
uestion-answering-over-csv-data/
… YouTube: https://
youtube.com/watch?v=jGnf4O
hptbA
… Code & data used: https://
github.com/langchain-ai/l
angchain-benchmarks
… Now for a quick thread: -
Improving AI Agents for CSV Question-Answering Applications
By
–
This started ~2 weeks ago, when I tweeted that we wanted to improve our chains/agents for doing question/answering over CSV data Why? Most QA applications focus on text data, but lots of real world data is in CSVs
-
Building AI Evaluation Datasets from Real User Questions
By
–
The first issue to solve was that we didn't have a dataset to evaluate Rather than try to make up some questions, we put out an example application and logged what questions people asked They also kindly provided feedback, so we could easily identify errors!
-
Benchmarking Question Answering Over CSV Data
By
–
Benchmarking Question/Answering Over CSV Data Deep dive on improving an application that does question answering over CSV data: 3000 word blog post
30min video
Open sourced eval data
Open sourced code for gathering feedback
Open sourced final agent code -
Agents with Multiple Retrievers for Conversational QA
By
–
this is pretty easily doable – you can give an agent access to multiple retrievers! https://
python.langchain.com/docs/use_cases
/question_answering/how_to/conversational_retrieval_agents
… -
ParentDocumentRetriever balances chunk size for LLM optimization
By
–
We introduced the ParentDocumentRetriever last week to strike a balance between: – using small chunks during indexing
– passing larger chunks to the LLM Great overview and diagram by @clusteredbytes -
Apps Embracing Open Source Models for Better Integration
By
–
Love to see more apps adding support for open source models!
-
Speaking at AWS Gen AI Day on Platform Integrations
By
–
Excited to be speaking at AWS Gen AI day! Lot's of awesome integrations with AWS platform, was fun to highlight them all https://
genaiday.virtual.awsevents.com/register?trk=5
7dbe57a-49c3-432b-a93b-0988d5011812&sc_channel=sm
…