Semantically annotating 3D gaussian splats on the fly using gemini 3.1 + sparkjs

1. Load any 3D scene and hit scan
2. Get 2D detections from VLM
3. Cluster outputs & project into 3D world space
4. Save as a persistent 3D semantic layer

Inspired by @alexanderchen's experiments… pic.twitter.com/CKZzpRkKtp

— Bilawal Sidhu (@bilawalsidhu) 15 mai 2026

Semantically annotating 3D gaussian splats on the fly using gemini 3.1 + sparkjs 1. Load any 3D scene and hit scan
2. Get 2D detections from VLM
3. Cluster outputs & project into 3D world space
4. Save as a persistent 3D semantic layer Inspired by @alexanderchen
's experiments

→ View original post on X — @bilawalsidhu