"OpenAI said before that evaluating their dataset was one of the major factors for the huge jump from ChatGPT3.5 to 4. Let's do the same for our own applications!" @SimonNom1 expressed better than I could why we're focusing so much on end-to-end evaluations!
@hwchase17
-
Amazon Bedrock LLMs Integration Represents Major Cloud AI Development
By
–
Amazon bedrocks LLMs are a big integration!
-
Custom Agent with Python REPL and Retriever Tools
By
–
Then we could go about improving it! Our final solution was a custom agent with two tools: – Python REPL (for doing analysis)
– Retriever (for dealing with text data present) We're excited to see other's solutions for this as well! -
LangChain Benchmarking Question-Answering CSV Data Tasks
By
–
Recap of important links: Blog: https://
blog.langchain.dev/benchmarking-q
uestion-answering-over-csv-data/
… YouTube: https://
youtube.com/watch?v=jGnf4O
hptbA
… Code & data used: https://
github.com/langchain-ai/l
angchain-benchmarks
… We had a lot of fun doing this and learned a lot – we're going to do it for more tasks! Up next: SQL -
LLM-Assisted Evaluation: Best Practice for Answer Assessment
By
–
Once we had these datapoints, we needed a way to evaluate answers For this, we relied on LLM assisted evaluation Although this isn't perfect, we think its the best thing out there and are bullish on this in the long run
-
Improving AI Agents for CSV Question-Answering Applications
By
–
This started ~2 weeks ago, when I tweeted that we wanted to improve our chains/agents for doing question/answering over CSV data Why? Most QA applications focus on text data, but lots of real world data is in CSVs
-
Building AI Evaluation Datasets from Real User Questions
By
–
The first issue to solve was that we didn't have a dataset to evaluate Rather than try to make up some questions, we put out an example application and logged what questions people asked They also kindly provided feedback, so we could easily identify errors!
-
LangChain Benchmarking Question Answering CSV Data
By
–
Blog: https://
blog.langchain.dev/benchmarking-q
uestion-answering-over-csv-data/
… YouTube: https://
youtube.com/watch?v=jGnf4O
hptbA
… Code & data used: https://
github.com/langchain-ai/l
angchain-benchmarks
… Now for a quick thread: -
Benchmarking Question Answering Over CSV Data
By
–
Benchmarking Question/Answering Over CSV Data Deep dive on improving an application that does question answering over CSV data: 3000 word blog post
30min video
Open sourced eval data
Open sourced code for gathering feedback
Open sourced final agent code -
Agents with Multiple Retrievers for Conversational QA
By
–
this is pretty easily doable – you can give an agent access to multiple retrievers! https://
python.langchain.com/docs/use_cases
/question_answering/how_to/conversational_retrieval_agents
…