I know you don’t talk a lot about them but I’d be curious to know your stance on closed-source models one day Alex Like where do you think models that are being integrated everywhere today like GPT-4, Bard or Claude are standing on the scale of copyright compliance?
LLMS
-
LLM Documentation: Langchain and OpenAI API Resources
By
–
Above will cover deep learning in general. For specific LLMs, documentation of the Langchain/OpenAI API is easiest.
-
Data Quality Emerges as Key Differentiator in AI Development
By
–
Hidden amongst the zingers in the leaked Google memo there's this: data quality is what is making the difference now. This is an important shift away from internet-scale garbage dump datasets that have been running the show for years.
-
Proper Task Prompts Essential for Fair Model Evaluation
By
–
Best is to evaluate the models with a prompt where the understand the task. If a model is 20% below with a prompt compared to another, clearly you are not evaluating the model properly and it makes no sense to report this number or use it in a comparaison.
-
Prompt engineering investigation: minimal sensitivity to formatting changes
By
–
We spent a lot of time investigating what others papers did (Chinchilla, GPT-3, PaLM) but very few of them actually provide any prompt so we just implemented what made sense to us. And we did not observe 20% differences by adding or removing a space in the prompt.
-
Custom Prompts and Benchmark Evaluation Standards for LLMs
By
–
You mention our "custom prompt" like if there was an official way of prompting (yours?). Most benchmarks were created before this concept of LLM eval with prompting even exists, and for many of them there is no official prompt or way to evaluate them with LLMs.
-
Formatting Alone Cannot Explain Performance Gap with LLaMA
By
–
I don't think that formatting alone can explain such a gap. You could maybe try to reproduce LLaMA numbers given the available model. It is likely that fixing these differences for LLaMA will also improve the numbers you report for your model.
-
LLaMA evaluation metrics concern and measurement discrepancies
By
–
It is true that accuracy on some metrics can be quite sensitive to the prompt, however this is not normal that all metrics reported for LLaMA here are systematically (and significantly) below what we measured. There may be an issue in how LLaMA was evaluated.
-
ChromaDB Self-Querying Retriever with LLM Integration
By
–
ChromaDB Self-Querying Retriever Last week we introduced the self-querying retriever Basic idea is to use an LLM to turn a user query into a "query" and a "filter" We now implemented to work with @trychroma
! Docs: https://
github.com/hwchase17/lang
chain/blob/master/docs/modules/indexes/retrievers/examples/chroma_self_query_retriever.ipynb
… -
ChatGPT surpasses doctors in empathy according to JAMA study
By
–
Pensez-vous que nous sommes à un tournant du monde médical ? Mes confrères sont stupéfaits Une étude dans la grande revue médicale #JAMA montre que #ChatGPT est BEAUCOUP plus empathique que nous médecins Et cette étude a été réalisée avec GPT3 et non GPT4 Votre avis ?