This sounds a bit more humble than the recent 1M or 1B context lengths claims. But this requires only minimal (1000 steps) finetuning. And it allows long document summarization using LLaMA 7B and 65B models, for example. Link to the paper here: https://
arxiv.org/abs/2306.15595 3/3
AI
-
Humble approach to long context lengths with minimal finetuning
By
–
-
Extending Transformer LLMs Context Window via Positional Interpolation
By
–
In the last couple of days, we talked a lot about extending the context window of transformer LLMs. Here's one more: "Extending Context Window of Large Language Models via Positional Interpolation" 1/3
-
Celebrity PR boosts competitor launch in tech rivalry
By
–
watching the worlds biggest celeb give free PR to the man who just happened to launch a competitor
-
AI Democratizes Healthcare by Providing Everyone Access to Best Doctors
By
–
How To Democratize Healthcare: AI Gives Everyone The Very Best Doctor
#AI #AIio #BigData #ML #NLU #Futureofwork @TunstallAsc @StrategyFintech @TamaraMcCleary @TerenceLeungSF @psb_dc @thomaspower @vinod1975 @ylecun http://
ow.ly/8VTC30sw8oT -
Upload Model Weights to Hugging Face Community Hub
By
–
Please consider uploading the weights to http://
hf.co -
Open Source LLM App Ideas Community Showcase
By
–
We can't wait to see what you build with it. "Give me some ideas for apps that can be built with an open source LLM."
-
Impressive Open Source LLM Performance with Creative Poetry Generation
By
–
The results are very impressive. You can quickly see why it's the best performing open source LLM. "Write a poem about open source machine learning in the style of Byron."
-
Falcon-40b-instruct Tops OpenLLM Leaderboard Rankings
By
–
Falcon-40b-instruct is the top performing model on the OpenLLM leaderboard. Built by @TIIuae
, it's free to use for commercial and research use. https://
huggingface.co/spaces/Hugging
FaceH4/open_llm_leaderboard
… -
Automation History: Humans Innovate New Production Forms
By
–
the means of production has already been 98% automated multiple times in human history. humans just start producing new things that their ancestors wouldn’t recognize as production