Special tokens like these are used to delimit things like "this section is the user asking a question" vs "this section is an answer from the LLM" – or sometimes for things like "this is the output of a separate tool" I have no idea what those gh ones are for!
@simonw
-
Understanding Tokenizers: The Vocabulary of AI Models
By
–
I wrote this explainer on tokenizers last year: https://
simonwillison.net/2023/Jun/8/gpt
-tokenizers/
… Effectively they're the vocabulary used by the model -
AI Models Remain Vulnerable to Adversarial Attacks
By
–
Sadly not: that paper concludes with "Finally, our current models are likely still vulnerable to powerful adversarial attacks." More of my notes here:
-
Claude 3 Opus and Meta.ai compete for LLM dominance
By
–
Claude 3 Opus from Anthropic is a genuine competitor now FOR BASIC LLM stuff, it's very worth trying out https://
meta.ai is free and does the integrated search thing as well as or even slightly better than ChatGPT web browsing -
Running ONNX Models on macOS Successfully
By
–
This ONNX one should run nicely on macOS if we can figure out the necessary incantations
-
Running 128K Token Model on Mac: Technical Setup Guide
By
–
Anyone got a working recipe for running the 128,000 token variant on a Mac?
-
Impressive Performance of Compact 2.2G Model File
By
–
First impressions: this thing is wildly impressive for a 2.2G model file
-
Phi-3 Language Model Released with 4K and 128K Context Options
By
–
Phi-3 is out – I just got the 4,000 token context model running using llamafile from the instructions in their README https://
huggingface.co/microsoft/Phi-
3-mini-4k-instruct-gguf
… They have a 128,000 model too! It's not available as a GGUF yet -
Llama 3, AI Data Journalism, LLM Evaluation Tools
By
–
Weeknotes: Llama 3, AI for Data Journalism, llm-evals and datasette-secrets