Eventually tracked down the problem to the Unsloth monkey-patching. Distributed runs even with one process fail, but if you use the regular models from transformers and `pef`, trained with `trl` then it doesn't deadlock… Hmm.
@alexjc
-
European RL Variant Focuses on Data Protection Privacy
By
–
Me: Did you know there's a European RL variant that focuses on data protection and privacy? Nobody: Oh, what's it called… Me: GDRPO!
-
NCCL Timeout Issue with Qwen3 LLM Engine Initialization
By
–
`accelerate test` works with the default config… But the script above hangs for 10 minutes likely due to the NCCL timeout. Somewhere between these two log lines: INFO [core(.)py:74] Initializing a V1 LLM engine (v0.10.1.1) with config: model='unsloth/Qwen3-4B-Base'
INFO -
Training Qwen3 Multi-GPU Setup with Accelerate Library
By
–
Anyone manage to get Qwen3 to train on multiple GPUs using `accelerate`? With a fresh install, I can't get even SFT to work with two 3090s.@UnslothAI The demo notebooks are nice, but a reproducible script known to work would be great, ideally with version dependencies pinned! https://t.co/8FkFpC3pdO
— Alex J. Champandard 🌱 (@alexjc) 10 septembre 2025Anyone manage to get Qwen3 to train on multiple GPUs using `accelerate`? With a fresh install, I can't get even SFT to work with two 3090s. @UnslothAI The demo notebooks are nice, but a reproducible script known to work would be great, ideally with version dependencies pinned!
-
Concatenative Languages and RL Environment Limitations
By
–
RQ6: Where best apply a concatenative language like Joy? Most datasets are rather static! Even many "RL" environments are single-turn, simple wrappers around static datasets that only allow the model space to think about a unique answer. The authors of the paper I linked
-
Cython Interpreter Optimization Through Memory Management Insights
By
–
In the meantime, I spent a few days looking into memory management and reference counting, which led to many insights about improving and reducing allocations, whole program optimization, language design, and ultimately another rewrite of my Cython interpreter. (Approx 15% faster
-
Joy Language: Function Composition for LLM Token Processing
By
–
RQ5: Which specific benefits does Joy bring to LLMs by processing tokens? Let's dig into the juicy bits about the primary "raison d'être" for the project… • Joy stands out as a language because it allows function composition by the sequencing of tokens (called "terms").
-
joyfl v0.2 Debug Information Language Update
By
–
joyfl — v0.2 : Debug Information Quality of life improvements on the surface for error handling, minor changes in the language design, one performance improvement. ▶︎ Condition-checking operators are suffixed with ?, ops with side-effects are suffixed with !.
▶︎ -
Token Origin Tracking and Fair Trade Implementation Updates
By
–
Now tracking the origin of each parsed token to ensure it's organic and Fair Trade, and the location of tokens is shown in better error messages… Many other changes today, will likely make the next release tomorrow!
-
Lark Parser: Handling Token Precedence in Bottom-Up Parsing
By
–
The parser I used is a library called Lark. It was relatively easy to get up and running, except for the implicit precedence of the tokens! Lark's algorithm is bottom-up: it parses tokens first then figures out where they fit. If your tokens are ambiguously defined (e.g. a