Build a Large Language Model from scratch! This repository contains the code for developing, pretraining, and finetuning a GPT-like large language model. 100% Free & Open Source
@sumanth_077
-
Axolotl: YAML-Based Fine-Tuning Framework with Memory Optimization
By
–
4. Axolotl • Yaml-based setup for fine-tuning, LoRA/QLoRA, DPO, GRPO, and multimodal workflows
• Includes kernel optimizations for memory-efficient training GitHub repo → -
LLaMA Factory: Fine-tune 100+ Models with LoRA and QLoRA
By
–
2. LLaMA Factory • Fine-tune over 100 models (LLaMA, Mistral, Gemma, etc.) using a simple CLI or WebUI
• Supports LoRA, QLoRA, full or frozen fine-tuning across 2–8‑bit precision GitHub repo → -
DeepSpeed: Distributed Fine-tuning Framework for Large-Scale LLM Training
By
–
3. DeepSpeed • Built for large-scale distributed fine-tuning with ZeRO and FSDP
• Optimized for multi-GPU and multi-node training with advanced memory management
• Trusted in production environments for scalable LLM training GitHub repo → -
Unsloth AI: Fine-tune LLMs 2× faster with 70% less VRAM
By
–
1. Unsloth AI • Fine-tune models like Qwen3, Llama 4, and Gemma 3 up to 2× faster with 70% less VRAM
• Supports low-resource setups and runs on consumer GPUs or even Colab/Kaggle with ~3 GB VRAM GitHub repo → -
Four Open-Source Libraries to Accelerate LLM Fine-Tuning
By
–
4 libraries that accelerates fine-tuning of LLMs (100% open-source)
-
RAGFlow: Open-Source RAG Engine for Deep Document Understanding
By
–
RAG engine for deep document understanding! RAGFlow is an open-source RAG engine for deep document understanding and streamlined knowledge workflows from complex data formats. 100% Open Source
-
Agentic Doc: Python Library for Document Data Extraction
By
–
Turn documents into LLM-ready data! Agentic Doc is a Python library for agentic document extraction. It pulls structured data from visually complex documents like tables, images, and charts and returns a hierarchical JSON with exact element locations. 100% Open Source
-
AI Agent Tutorials and Pydantic-Based Agentic Applications
By
–
I've created AI Agent tutorials and more in this repo. I'll be building Agentic Applications using Pydantic. 100% open source. Check out the repo and don't forget to leave a star:
-
PydanticAI: Open Source Python Framework for Production Agentic AI Applications
By
–
Build production-grade Agentic AI apps in pure Python! PydanticAI is a Python agent framework designed to simplify building production-grade Agentic applications. 100% Open Source