Retrieval systems often split long documents into chunks, but this strips away the surrounding context. Contextual embedding models fix this by encoding the whole document once and pooling chunk vectors afterward. They are usually trained on one gold chunk per query.

→ View original post on X — @perplexity_ai