Freezing all layers except these: – ^lm_head.weight$ – ^model.embed_tokens.weight$ – model.layers.4[4-9]+.block_sparse_moe.gate – model.layers.4[4-9]+.block_sparse_moe.experts – model.layers.5[0-5]+.block_sparse_moe.gate – model.layers.5[0-5]+.block_sparse_moe.experts
OPEN SOURCE
-
CUDA kernels for GPT-2 forward pass implementation in llm.c
By
–
Okay I did a first quick pass of naive CUDA kernels for the forward pass of GPT-2 and pushed everything to one file in llm.c, Still only ~1000 lines of code: https://
github.com/karpathy/llm.c
/blob/master/train_gpt2.cu
… Current per iteration timings on my Lambda box <3 A100 40GB PCIe, B=4, T=1024:
– llm.c: 111ms
– -
LangChain Contribution Guide: Join Open Source AI Development
By
–
Thank you all so much! If you'd like to contribute something yourself, give our contribution guide a read: https://
python.langchain.com/docs/contribut
ing/
… (6/6) -
LangChain Integrations: MLX, Solar Models and HTML Splitter
By
–
@Prince_Canuma for adding an @Apple MLX model integration https://
python.langchain.com/docs/integrati
ons/chat/mlx/
… @m_setayesh for adding a new "section-aware" HTML text splitter https://
python.langchain.com/docs/modules/d
ata_connection/document_transformers/HTML_section_aware_splitter/
… Tokkiu for adding @upstageai Solar model and embeddings
https://
python.langchain.com/docs/integrati
ons/chat/solar/
… (3/n) -
LangChain Tools: Haskell Code Splitter and Financial Data Integration
By
–
@nisargtrn for a text splitter specializing in Haskell code
https://
python.langchain.com/docs/modules/d
ata_connection/document_transformers/code_splitter/
… anshaneel for an @alpha_vantage
-powered financial data tool usable in agents
https://
python.langchain.com/docs/integrati
ons/tools/alpha_vantage/
… (4/n) -
Community Spotlight: Top Python Contributions to LangChain
By
–
Community PR roundup We get fantastic community PRs to LangChain daily, and we'd like to spotlight 10 Python contributions from the past three weeks! In no particular order, a huge thank you to: (1/n)
-
LangChain adds reranking and token counting integrations
By
–
kennethchoe for adding reranking based on @HuggingFace cross encoder models https://
python.langchain.com/docs/integrati
ons/document_transformers/cross_encoder_reranker/
… dmenini and Sukitly for adding token counting to @Anthropic models on @Amazon Bedrock (2/n) -
Meta Aligns PyTorch Compiler Flow on Custom AI Hardware
By
–
+1 to same compiler flow on our chips as mainline PyTorch. At Meta, we don't have a separate copy of PyTorch or Dynamo/Inductor. The source of truth is Github and everything is upstreamed/mainline. Triton's bugs have been going over time, but we liked it as a starting point, and
-
Triton Backend Enables AMD Code Generation Support
By
–
Triton can generate anything, as long as you write a Triton backend. As of today, upstream Triton can generate AMD code too.
