MACHINE LEARNING

AI knows URLs from training data, cannot browse web

By

–

03 December 2022 5h45

I assure you it can't browse the web, it just knows the URLs for many, many websites. It's seen many URLs in its training data, and can both recite real ones and hallucinate plausible fake ones.

Sliding window of 8192 tokens confirmed in latest update

By

–

03 December 2022 5h16

See my update in the replies — I now believe it's simply a sliding window of length 8192 tokens.

ChatGPT uses a simple sliding window of 8192 tokens

By

–

03 December 2022 4h31

I now believe ChatGPT uses a simple sliding window with a context window of 8192 tokens:

ChatGPT context length 8192 tokens inferred from repetition test.

By

–

03 December 2022 4h30

Update: Recollection of the start of the conversation reliably fails after 7 repetitions of the ~1024-token input. This suggests that text-davinci-002-render, the model underlying ChatGPT, has a likely context length of 8192 tokens, with some tokens consumed by formatting.

ChatGPT recalls content beyond 4096-token context window

By

–

03 December 2022 4h00

A demonstration that ChatGPT is able to recall the content of conversations beyond the length of a 4096-token context window. Uses four messages of 1024+ tokens, each ending with instructions to demonstrate that input is not truncated before the model sees it.

→ View original post on X — @animaanandkumar

Falcon: Model-Based RL for Extreme Turbulent Environments

By

@animaanandkumar

–

03 December 2022 1h20

https://t.co/EGIj76FI28
— Prof. Anima Anandkumar (@AnimaAnandkumar) 3 décembre 2022

Tune in for our talk on Falcon, first model-based RL method in real-world extreme turbulent environments (Reynolds Num. 230000!), AI4Science workshop @ #Neurips2022, today @ 1:20pm. Falcon achieves 37% improvement over SOTA in stabilization of an airfoil using only 8 min of data!

→ View original post on X — @datachaz

Building Intent Classifier with Streamlit and spaCy

By

@datachaz

–

03 December 2022 1h13

Asking #ChatGPT to 'code an intent classifier with @Streamlit and @spacy_io
' It needs tweaking a bit, yet adding that to some docs, tutorials or a blog post would be totally fine by me! cc @jrieke