Special tokens like these are used to delimit things like "this section is the user asking a question" vs "this section is an answer from the LLM" – or sometimes for things like "this is the output of a separate tool" I have no idea what those gh ones are for!
GENERATIVE AI
-
Understanding Tokenizers: The Vocabulary of AI Models
By
–
I wrote this explainer on tokenizers last year: https://
simonwillison.net/2023/Jun/8/gpt
-tokenizers/
… Effectively they're the vocabulary used by the model -
AI Models Remain Vulnerable to Adversarial Attacks
By
–
Sadly not: that paper concludes with "Finally, our current models are likely still vulnerable to powerful adversarial attacks." More of my notes here:
-
Claude 3 Opus and Meta.ai compete for LLM dominance
By
–
Claude 3 Opus from Anthropic is a genuine competitor now FOR BASIC LLM stuff, it's very worth trying out https://
meta.ai is free and does the integrated search thing as well as or even slightly better than ChatGPT web browsing -
LLM Safety: Instruction Hierarchy Defense and Team Hiring
By
–
LLMs process text from multiple sources and may face conflicting instructions. We teach our models to follow instructions from the highest priority input, giving better defense against attacks. Our Safety Systems team is hiring:
-
Meta’s 15T Token Training Strategy Preference
By
–
would prefer they just trained on 15T tokens like meta
-
Gemini 1.5 Pro: Model size specifications and capabilities
By
–
what do you mean by "it's class"?
Did Gemini 1.5 Pro's size ever get revealed? -
Lutra AI Gains Self-Reflection and Faster Capabilities
By
–
@Lutra_AI is getting easier to use and more capable! We've been hard at work bringing in more capabilities for it to self-reflect, self-debug, and run even faster — so that users can just give it high level directions and it'll figure it out. As models accelerate in
-
OpenAI launches enterprise-grade API features with enhanced security controls
By
–
We’ve introduced more enterprise-grade features for API customers, including enhanced security, administrative controls, new Assistants API capabilities, and tools to help better manage costs.