AI - AI Dynamics

Proper Task Prompts Essential for Fair Model Evaluation

By

–

05 May 2023 23h37

Best is to evaluate the models with a prompt where the understand the task. If a model is 20% below with a prompt compared to another, clearly you are not evaluating the model properly and it makes no sense to report this number or use it in a comparaison.

→ Voir le post original sur X — @dfintelligence

Training an AI voice model to sing

By

@dfintelligence

–

05 May 2023 23h37

Pour ceux qui croit qu’il suffit de cliquer sur un « bouton » pour avoir ce résultat allez voir la vidéo YouTube https://
youtu.be/ECIas_koYcI J’ai codé et testé des jours, en plus de dépenser des sous, pour entraîner un modèle de voix de Macron qui me permet de le faire chanter.

Prompt engineering investigation: minimal sensitivity to formatting changes

By

–

05 May 2023 23h35

We spent a lot of time investigating what others papers did (Chinchilla, GPT-3, PaLM) but very few of them actually provide any prompt so we just implemented what made sense to us. And we did not observe 20% differences by adding or removing a space in the prompt.

Custom Prompts and Benchmark Evaluation Standards for LLMs

By

–

05 May 2023 23h34

You mention our "custom prompt" like if there was an official way of prompting (yours?). Most benchmarks were created before this concept of LLM eval with prompting even exists, and for many of them there is no official prompt or way to evaluate them with LLMs.

Formatting Alone Cannot Explain Performance Gap with LLaMA

By

–

05 May 2023 22h43

I don't think that formatting alone can explain such a gap. You could maybe try to reproduce LLaMA numbers given the available model. It is likely that fixing these differences for LLaMA will also improve the numbers you report for your model.

→ View original post on X — @kirkdborne

Storage Platform Role in Enterprise AI and Analytics

By

@kirkdborne

–

05 May 2023 22h35

#StorageMatters — yes, it does! and you can learn all about Storage Matters and how the right storage platform can accelerate your enterprise AI and Analytics initiatives — Register now to attend the big @PureStorage Accelerate 2023 event in June in Las Vegas:

LLaMA evaluation metrics concern and measurement discrepancies

By

–

05 May 2023 22h19

It is true that accuracy on some metrics can be quite sensitive to the prompt, however this is not normal that all metrics reported for LLaMA here are systematically (and significantly) below what we measured. There may be an issue in how LLaMA was evaluated.

→ View original post on X — @dominodatalab

LGIM Partners with Domino for Data-Driven Investment Model Generation

By

@dominodatalab

–

05 May 2023 22h00

Proud to partner with LGIM in their data-driven transformation! Excited to see our platform accelerate model generation in investments. Thanks, LGIM! Read more:
https://
domino.buzz/3nABFFK] #DataScience #LGIM