Prompt-engineering mitigations to prompt echo attacks are not robust. The only 100% reliable defense is to use a fine-tuned model, tuned without instructions in the examples. No prompt, no problems.
MACHINE LEARNING
-
Content Summarization and Sentiment Analysis in NLP
By
–
In this @Cognilytica #AIToday #podcast episode 'Content Summarization and Analysis, Sentiment Analysis' from the AI Glossary Series, hosts @rschmelzer & @kath0134 define Content Summarization and #Analysis and #sentimentanalysis. Full episode: https://
cognilytica.com/2022/12/28/ai-
today-podcast-ai-glossary-series-content-summarization-and-analysis-sentiment-analysis/?utm_source=dlvr.it&utm_medium=twitter
…
#ML #AI #nlp -
Word embeddings and latent factors in language models
By
–
I'd argue that the janky "queen = king – man + woman" thing is definitely related. It's not doing language generation, but you're trying to query the latent factors the model has used to minimise the surface objective
-
Using nlp.pipe() for faster batched predictions in spaCy
By
–
Nice example! Here's a tip for next time as well: the nlp.pipe() function operates over a sequence of inputs, and lets the model make batched predictions, making it much faster:
-
Neural Radiance Fields (NeRF) for Unconstrained Photo Collections
By
–
Paper: Neural Radiance Fields (NeRF) for Unconstrained Photo Collections https://
nerf-w.github.io -
Conceptual Principles of Machine Learning and Computer Vision
By
–
137 pages of very conceptual principals around Machine Learning, Object Detection, and Computer Vision. Here's some early concepts of pattern recognition, building descriptions, and enforcement:
-
LLMs like Copilot replace Google for code installation commands
By
–
Imports and installation commands are the purest instance of LLMs replacing Google for me. Only the comments in this screenshot were written by hand, and the rest is Copilot completion:
-
Google’s PaLM benchmark results and open-source RLHF implementation
By
–
Google's PaLM beats GPT-3 on 28 out of 29 Natural Language Processing tasks (!) and was just open-sourced for reinforcement training as of today: https://
github.com/lucidrains/PaL
M-rlhf-pytorch
… -
Meta AI OPT-IML Benchmark Improves LLM Instruction Tuning
By
–
Improving Instruction Tuning for LLMs: Meta AI Presents the OPT-IML Benchmark of 2000 NLP Tasks
-
FlashAttention Revert and torch.compile Integration Challenges
By
–
Ah, I reverted FlashAttention in this run because it made code messier. Will look into incorporating it back, but yes not sure how nicely it plays with torch.compile. The usual problem with taking on large dependencies you don't understand ;(