That's a different issue for me – I want scores for the tasks that are most useful for document analysis with LLMs
LLMS
-
Best LLM Benchmarks for Summarization and RAG Tasks
By
–
LLM benchmark question: benchmarks like MMLU do a lot of testing for knowledge – what are the most interesting benchmarks for if I don't care as much about what the model "knows" but more about how good it is at tasks like summarization, data extraction and RAG Q&A against input?
-
Custom AI Models and LLM Accuracy Through Quality Training Data
By
–
This! The accuracy of custom AI models and private LLMs is a direct reflection of the training data. We pioneered data-centric AI and programmatic data development to help enterprises accelerate creation of high quality training data. Get the full story: https://
buff.ly/3xEws4u -
Novelty in MSJ Paper: Context Length Impacts AI Mitigation Effectiveness
By
–
To be fair the claimed novelty in the MSJ paper isn’t k-shot jailbreaking itself, it’s that tuning mitigations sufficient in the short-context regime are ineffective when k is large
-
Gemini 1.5 Pro In-Context Learning Scaling Techniques
By
–
It's been quite interesting to study how Gemini 1.5 Pro scales its in-context learning (ICL) from few to many shots. I found our experiments that avoid using hand-labeled examples with Reinforced ICL, and Unsupervised ICL (i.e. shots are input examples only), particularly neat.
-
7 ChatGPT prompts that make you a superhuman
By
–
7 ChatGPT prompts that'll turn you into a superhuman: (Bookmark this for later)
-
AI Policy: Balancing Competition, Security, Privacy and Sustainability
By
–
Policy makers should consider the interactions between AI, data, competition law, security/privacy & carbon/ climate/ energy policy. There are emerging techniques that help address these issues such as Federated Learning w/Differential Privacy;
RAG w/LLMs;
Context based -
GPT-4 Training Cost Estimates Questioned
By
–
most estimates have gpt4 all-in cost at least 9 digits. idk how they calculated the compute
-
Training LLMs: Avoiding Default Mode Through Data Curation
By
–
First you’d teach the model quite a bit from recent content (w/ tons of LLM outputs in it it’s likely, if we were to train this in normally, the LLM will likely get stuck in that default LLM ‘mode’ we know so well + make it harder to break out of this w/ post-training) So
-
Training Strategy: Pre-2021 Data Priority Over AI-Generated Content
By
–
With all the AI-generated content flooding the web There might be something to first training on content from 2021-on And then continuing to train on pre-2021 content