Have you explored how this would be affected by low-rank pretraining approaches such as Galore? https://
arxiv.org/abs/2403.03507
CODE
-
Galore Low-Rank Pretraining Impact on Model Training
By
–
-

Keras Models Publishing to Hugging Face Hub Integration
By
–
Publish your models directly from Keras to @huggingface I've been wanting to do this for the longest time – basically for as long as i've used Keras. (Fun fact: our chatbot back in 2016 had many parts that were powered by Keras.) The ability to natively load from/to the Hub
-
Enterprise AI Observability and Open-Source LLM Hosting Launch
By
–
ICYMI: We launched new cross-environment AI observability with enterprise-grade open-source LLM hosting — available across all AI assets and environments. Not able to join live? Check out the on-demand recording from our virtual launch event with @decisionleader and @nvidia
-
Using AI to pipe and combine existing code for new tools
By
–
I have a lot of fun piping in existing code and having it make changes – adding small new features for example like here, where I gave it a bunch of my existing code and had it combine that to produce a new tool https://
simonwillison.net/2024/Mar/30/oc
r-pdfs-images/#ocr-how-i-built-this
… -
Using LLMs to Solve Coding Problems Quickly
By
–
This one is a particularly good illustration of working in conjunction with an LLM to very quickly solve a small but interesting coding problem https://
simonwillison.net/2024/Mar/22/cl
aude-and-chatgpt-case-study/
… -
Exploring LLMs for Code Development Notes and Resources
By
–
Here are other notes I've made about exploring LLMs for code over the past couple of years https://
simonwillison.net/series/using-l
lms/
… -
Using LLMs Responsibly: Code Review and Quality Assurance
By
–
As with so many things involving LLMs, combining with deep existing expertise is important – to responsibly use them for code you need to be great at reading code, code review, considering edge-cases, automated testing and extensive QA Good code review skills go a long way here
-
Code Generation: AI’s Self-Verifying Use Case
By
–
I actually think code is one of the most responsible use-cases for it, because you get a weird form of fact checking for free: if it hallucinates methods that don't exist you find out the moment you actually try to run the code and it fails Much easier to verify code than prose!
-
Using ChatGPT to Build C Extensions for SQLite
By
–
I use it for code all the time – here are some of my most relevant notes: https://
simonwillison.net/2024/Mar/23/bu
ilding-c-extensions-for-sqlite-with-chatgpt-code-interpreter/
… and -
Auto-sync nodes to latest version without manual drag-drop
By
–
2/3
— FlowiseAI (@FlowiseAI) 7 mai 2024
Automatically sync nodes to latest version without having to re-drag & drop. pic.twitter.com/f04aFWbrge2/3 Automatically sync nodes to latest version without having to re-drag & drop.