this is neat. Anthropic never released the tokenizer for Claude 3 (I'm not sure why?). to tokenize a string, you can just ask Claude to repeat it, and store the tokens as they're generated. probably the most computationally expensive tokenizer ever created, but it works
@jxmnop
-
Daily Deep Learning Workout: CNNs, RNNs, and CUDA Kernel Training
By
–
daily deep learning workout: train two CNNs and three RNNs. perform ten minutes of quantized LLM transformer inference. write CUDA kernels until failure
-
Extracting Full Logits from OpenAI API via Binary Search
By
–
for those who haven't heard about this:
openAI only gives you the top 5 logprobs per response (max). we (mostly @justintchiu
) came up with an algorithm to get logits from openAI. essentially for a different token (not top-5), u can binary search logit bias to see the minimal -
Language Model Inversion: New Research and Open Source Tools
By
–
paper (language model inversion) http://
arxiv.org/abs/2311.13647 github (open_logprobs) -
Language Model Inversion Attack Research Paper
By
–
language model inversion http://
arxiv.org/abs/2311.13647 -
OpenAI API Changes Due to Language Model Inversion Research
By
–
achievement unlocked: OpenAI changes its API because of your research you can still get logits from GPT-4 by using the argmax bisection method detailed in Language Model Inversion. it's pretty expensive, though we should have stockpiled logits while we had the chance…
-
Vec2text: Inverting Text Embeddings Through Stories
By
–
wrote a little blog post about our vec2text, our method for text embedding inversion! it's like the paper but told as a story: with a lot less jargon, and no math. check it out 🙂
-
GitHub Introduces Multiple README Files Feature
By
–
github lets you have multiple READMEs now. actually a really nice new feature pic.twitter.com/Jty2jhorVO
— dr. jack morris (@jxmnop) 4 mars 2024github lets you have multiple READMEs now. actually a really nice new feature
-
GPU Memory Requirements for Document Embedding Storage
By
–
I'd guess pretty poorly. but you really don't need that much GPU memory to store a fairly large corpus. all you need is the bag-of-words information for each document; that's the "embedding". eventually you will run out though, and I don't have any hierarchical logic built in.
-
BM25 Search Implementation Performance Comparison with ElasticSearch
By
–
also shout out to @clifapt for running BM25 via ES on LoCo first: https://
x.com/clifapt/status
/1755303087873429667
… my implementation seems to be slightly less performant than ElasticSearch; not sure why. maybe because i'm using a subword tokenizer, or suboptimal hyperparams?