I think we are talking about the same thing. That would only work after training. It's based on the assumption that the matrices have low rank for a specific target task but are not low rank in general (for the pretraining tasks).
INNOVATION
-
Multi-query attention experiences revival with Falcon model
By
–
Also, multi-query attention has a little comeback (thanks to Falcon)
-

Self-Attention Alternatives Failed; FlashAttention Gains Popularity
By
–
You mean alternatives to self-attention specifically (versus parameter efficient finetuning etc.)? I think none of them stood the test of time. I think the problem is that they are all approximations. The relatively recent FlashAttention mechanism is super popular though.
-

NLLB Browser Translation App Powered by Transformers.js
By
–
machine translation app using Meta's NLLB model that runs directly in your browser Powered by transformers.js by @xenovacom Space: https://
huggingface.co/spaces/julien-
c/nllb-translation-in-browser
… -

Dataset Importance for LLM Fine-tuning: Camels Paper Insights
By
–
And the recent "How Far Can Camels Go" paper (
https://
arxiv.org/abs/2306.04751) is a nice example of how important the dataset for LLM finetuning is. -

LLM Enhanced with Search and Symbolic Math Tools
By
–
The important part is when the LLM will be combined with search and symbolic math tool – the latter will enhance logical reasoning.
-

Mobile Banking: The New Financial Services Battleground
By
–
Has mobile become the battleground for banking? #Technology #Tech #FutureTech https://
computerweekly.com/opinion/Has-mo
bile-become-the-battleground-for-banking?utm_source=dlvr.it&utm_medium=twitter
… -
Newark schools test AI tutoring bot for student learning
By
–
Officials in Newark, New Jersey, are testing an AI enabled tutoring bot in three schools
-
AI Reduces Cancer Radiotherapy Wait Times for NHS Patients
By
–
Good use case: A new type of artificial-intelligence technology that cuts the time cancer patients must wait before starting radiotherapy is to be offered at cost price to all NHS trusts in England. #AI
-

Survey Shows 73% Pressure for AI Adoption in Enterprises
By
–
According to a global survey of 1,000 HR, finance, and IT decision-makers, 73% are feeling the pressure to increase AI adoption, but many don’t know how to best utilize the technologies. The report gives guidance on #AI #MachineLearning adoption