Sure I don’t disagree, but since all these were before RLHF finetuned decoder architectures, I am curious whether it’s still necessary to have an encoder module. LLaMA-Adapter would be an example of adding img2txt capabilities w/o cross attention.
GENERATIVE AI
-
ChatGPT won’t revolutionize programming as much as feared
By
–
@mikeloukides on how things like ChatGPT will change programming, but not as much as we may think https://
oreilly.com/radar/fearing-
the-wrong-thing/
… -
AI-Generated Travel Guides Lack Quality, Grammar on Amazon
By
–
'One, whose author is listed as Stuart Hartley, boasts, ungrammatically, that it is "Everything you Need to Know Before Plan a Trip to Paris." The book itself has no further information about the author or publisher. It also has no photographs or maps' https://
nytimes.com/2023/08/05/tra
vel/amazon-guidebooks-artificial-intelligence.html
… -
Decoder-Only Architectures for Vision-Language Models
By
–
That makes sense. But you could also use a decoder-only architecture (with embedded image tokens as part of the input, as in LLaMA-Adapter, for example). (* This uses an encoder for the tokens, but it's still a decoder-only architecture due to the lack of cross-attention)
-
Initial AI Implementation: Data Collection and Open Source Competition
By
–
The initial implementation is pretty basic. Part of this is also to see what types of questions people ask We’ll probably gather data, make some improvements, then open source the data plus eval and turn it into a little competition to improve
-
Improving Question Answering Systems for Tabular Data
By
–
CSV Question Answering Most QA applications deal with asking questions of unstructured data. But a lot of data is in tabular (excel, csv) format! This has always been tricky to get right. We want to improve that! But we need your help
-
CSV-LangChain App Lets Users Query Datasets Naturally
By
–
We've set an example app to ask questions of a toy CSV (the Titanic dataset, a classic data science dataset) Access the app here: https://
csv-langchain.streamlit.app How you can help: try to break it! Ask questions that you would want it to be able to answer, and then leave feedback -
Comparison of decoder models with ChatGPT and GPT-4 after RLHF
By
–
! I’m mainly curious how it compared to decoders like ChatGPT or GPT-4 after RHLF
-
Comparing Decoder-Only Models to Encoder-Decoder Architectures for Translation
By
–
The original transformer is an encoder-decoder arch for translation. T5 is a great encoder-encoder that’s pretty good at translation. ChatGPT / GPT-4 is a decoder-only that’s pretty good at translation too. How does it compare to encoder-decoder architectures of similar size?
-
Encoder vs Decoder LLMs: Benefits for Seq2Seq Tasks
By
–
Use cases for encoder LLMs (classification) & decoder LLMs (chatbots) are obvious. Seq2seq tasks like translation, where it makes sense to have access to the whole input, is where it gets interesting.
Re encoder-decoder LLMs (eg T5): Is there still a benefit of using an encoder?