What makes task arithmetic actually work? Researchers from Nanjing University, U. Wollongong, and NTU Singapore introduce OrthoReg: a simple method that enforces orthogonality in weight updates during fine-tuning. This promotes weight disentanglement and consistently boosts