Yeah lots of LLMs are trained on synthetic data these days, especially at the fine-tuning stage – but that's deliberate, the open question for me is still the impact of accidental synthetic data in the larger pre-training data pool
DATA
-
Smart data crawling strategy local iteration analysis
By
–
A dumb crawler doesn't necessarily mean that they're not doing something smarter with the content once it's been crawled – it's generally a good idea to crawl once and then iterate on analysis against your local copy
-
TMLR Paper with Experiments, Data and Code Resources
By
–
More experiments can be found in our TMLR paper: https://
arxiv.org/abs/2310.14540
Data and code: https://
huggingface.co/datasets/yyama
da/SpatialEvalLLM
…, https://
github.com/runopti/Spatia
lEvalLLM
…
Special thanks to @AndrewLampinen for helpful discussion and advice!
7/n -
Iterative Fine-Tuning Impact on Pre-Training Data Contamination
By
–
How much does deliberately iterative fine-tuning on model outputs tell us about the impact of smaller amounts of generated data making it into larger pre-training datasets?
-
Artificial Data Risk at Pre-training Stage in Models
By
–
That seems like a slightly different aspect of this to me – we have seen tons of examples now of models being fine-tuned on carefully created artificial data, but to me that doesn't speak to the risk of unintentional artificial data affecting the models at the pre-training stage
-
Data AI Summit Early Bird Registration Discount Ends April 30
By
–
Early bird catches the worm Save $400 by registering for #DataAISummit before April 30! You’ll explore the latest advances in #ApacheSpark, Delta Lake, MLflow, LangChain, PyTorch, dbt, and more: https://
bit.ly/3UlaDA7 -
Google DeepMind Synthetic Data Research Best Practices Overview
By
–
5/ Best Practices and Lessons on Synthetic Data – an overview by Google DeepMind on synthetic data research, covering applications, challenges, and future directions
-
DBRX: New Open-Source LLM Outperforms Established Models
By
–
#DBRX is a new general-purpose LLM that outperforms all established open-source models on standard benchmarks!
— Databricks (@databricks) 13 avril 2024
DBRX is incredibly efficient thanks to its fine-grained Mixture-of-Experts architecture. Learn how you can get started👇https://t.co/vtG4r2m5hY pic.twitter.com/9o5agDAJBk#DBRX is a new general-purpose LLM that outperforms all established open-source models on standard benchmarks! DBRX is incredibly efficient thanks to its fine-grained Mixture-of-Experts architecture. Learn how you can get started https://
dbricks.co/49g4ZDt -
Alteryx Analytics Cloud Google Integration Transforms Data Analytics
By
–
Unlock the power of data and analytics with Alteryx Analytics Cloud, seamlessly integrated with Google Cloud, for a transformative digital journey. Read about how Alteryx Analytics Cloud and Google Cloud work together on INPUT: https://
ow.ly/JSJ750Rf3EO #Analytics