Announcing Ludwig v0.8—the first #opensource low-code framework optimized for building and #finetuning LLMs on your data. New features incl. fine-tuning, integrations w/ Deepspeed, parameter efficient fine-tuning (#LoRA), prompt templating and more!
OPEN SOURCE
-
Open Source and Program Scale Reuse as Software Development Silver Bullet
By
–
I do think that open source and program scale reuse (vs function scale) turned out to be the closest thing to a silver bullet we got.
-
Hugging Face Launches TRL for RLHF Model Training
By
–
TRL Hugging Face Excited to announce that we're doubling down on our efforts to democratize RLHF and reinforcement learning with TRL, new addition to the @huggingface family, developed and led by team member @lvwerra Train your first RLHF model https://
github.com/huggingface/trl -
LangChainJS Text Splitters Available for Development
By
–
i think we have most of these splitters in langchainjs – @Hacubu can maybe help you out
-
Backdrop launches 100 Builders program for open source developers
By
–
We’re excited to support more people building open source tech @withBackdrop! 100 Builders is 100% remote, 100% free, and 100% on builders' terms. Apply now – applications close Aug 10!
-
Abacus AI Open-Sources Giraffe Long-Context Models
By
–
We are open-sourcing the models and the evaluation datasets and performance experiments. Git repository: https://
github.com/abacusai/Long-
Context
…. You can use the model directly from HuggingFace. 4k version: https://
huggingface.co/abacusai/Giraf
fe-v1-delta-13b-scaled-4
…. 16k version: https://
huggingface.co/abacusai/Giraf
fe-v1-delta-13b-scaled-16
…. -
ReLoRA Paper: Exploring LoRA for LLM Pretraining Beyond Finetuning
By
–
Just catching up with the ReLoRA paper (
https://
arxiv.org/abs/2307.05695) that explores whether LoRA can be used for pretraining LLMs (vs finetuning). Looks promising! Caveat: they pretrained models up to 350 M parameters (the smallest Llama model is 7 B parameters, for comparison) -
Candle Torch Rust Framework Preview Released
By
–
like a candy store opening in town https://
reddit.com/r/rust/comment
s/15lidhr/early_preview_candle_torch_in_rust/
… -
Adding multilingual code evals to BigCode evaluation platform
By
–
Nice! Would be cool to add it to https://
huggingface.co/spaces/bigcode
/multilingual-code-evals
…! -
ParentDocumentRetriever Implementation in LangChain Framework
By
–
I ended up going a slightly different direction (in terms of implementation): https://
github.com/langchain-ai/l
angchain/pull/8941
… ParentDocumentRetriever – retrieves the parent document for a given chunk. not exactly the same but close?