Deberta does work quite well for seq cls, but Mistral should generally outperform it. However in this case the target modules do not cover enough (it should be all linear layers in the main backbone) so there's *much* fewer trainable params
@jeremyphoward
-
ChatGPT Web Browsing Reluctance Issues Remain Unresolved
By
–
ChatGPT web browsing is still extremely reluctant to actually browse the web…
-
Platform Verification and Monetization Strategy for AI Content Creators
By
–
Yeah I got "verified" so they'd pay me, but removed the cringe-mark.
-
High-Quality ML/AI Discussion Migrates to Discord
By
–
Pretty much all the high-quality ML/AI discussion has moved to Discord now, so I don't think there's as much reason to follow me here as there used to be.
-
Journalism Quality and Credulity in AI Coverage
By
–
A lot of people in tpot are credulous morons, to be fair. Ideally, journalists would be neither of those things. (We don't live in an ideal world…)
-
GPT-4V threatens job displacement in professional sectors
By
–
Hey don't bring Levy in here — he'll be heartbroken when he discovers that GPT-4V is putting him out of a job…
-
Post-Elon X Platform Sees Surge in Obvious Bot Activity
By
–
Why do I see *far* more obvious bots in my replies now than I did in the pre-Elon days? I don't get it.
-
GPT-4V Chess Analysis Capabilities Questioned by Users
By
–
GPT-4V's chess analysis is kinda like watching actors pretend to play musical instruments in movies… Looks impressive, as long as you have no idea what's going on.
-
Making it Easier: New Session with Legal Move
By
–
OK let's make it easier for it; in a new session, we'll flip the board around, and we'll give it a legal move to consider: