@jeremyphoward

–

13 June 2025 21h53

ModernBERT is not a chat model.

Developer tools and creator tooling: do devs count as creators?

By

–

13 June 2025 20h47

Do devs count as creators? Because there’s a lot of dev tools companies that have done well. Actually I guess don’t really know what you mean by creator tooling, since I would have thought stuff like figma would count as tooling for creators?

Fine-tuning as continued pretraining improves medical AI

By

–

13 June 2025 20h40

Your daily reminder that fine tuning is just continued pretraining. Super cool results from @antoine_chaffin who is putting this knowledge into practice to improve medical AI:

Claude’s summarization approach in code implementation

By

–

13 June 2025 20h38

Of course – we use it a lot! Although afaict Claude code uses a basic summarization approach.

Fast.ai: A Powerful Foundation for Dedicated AI Learners

By

–

13 June 2025 5h20

http://
fast.ai can be a pretty amazing foundation for folks willing to put in the time and effort to learn, like @Suhail
, who worked really really hard.

Prefix Tuning Approaches: Layer-wise Activation Prepending Comparison

By

–

13 June 2025 3h14

My understanding is that @percyliang
's prefix tuning approach prepended activations to *every* layer, but yours only prepends to the first layer — is that right?

Prefix Tuning Initialization with Real Tokens and DoRA

By

–

13 June 2025 3h10

Is the trick of initializing with real tokens the secret to making all prefix tuning approaches competitive? BTW @EyubogluSabri I don't see DoRA mentioned in your paper – I wonder if that would close the gap a bit?

LoRA vs DoRA vs Prefix Tuning: Fine-tuning Method Effectiveness

By

–

13 June 2025 3h08

The big open question for me is why LoRA (and particularly DoRA) has been the most successful for fine-tuning, but it worked so badly for this use case. Or maybe the opposite – why did prefix tuning work so well?