Those asking how to do this, here's the recipe:
TL;DR wiggle the camera around while taking screen shots and asking Gemini for screen space annotations and then clustering them so they are pinned to the right spot in 3d space. You can make it even more precise to project into

→ View original post on X — @bilawalsidhu