Can you k-shot it with ground truth examples of photos and their depth maps? Like a 2-by-n grid of examples with the final image missing.
@goodside
-
ChatGPT conversation reveals fading role of knowledge cutoffs
By
–
I just got 10 questions deep in a chat with ChatGPT o3 on some finer points of MCP before it occurred to me it had never heard of MCP before our chat. Search is finally starting to work. In a few months nobody will know what a "knowledge cutoff date" even is.
-
Origin of the term “prompt injection”
By
–
Simon came up with the name “prompt injection”; also Jon Cefalu actually discovered it a few months before I did but that wasn’t public info at the time
-
Multimodal tool use with CoT and vision libraries
By
–
It’s tool use (PIL/cv2, BFS) debugged with multimodal perception of tool output within the CoT; it’s not 4o-style multimodal output. See the share link in the thread for more info on the tool calls.
-
Iterative multimodal debugging with Python vision tools
By
–
Yes; it seems to be tool use (PIL/cv2 in Python) iteratively debugged via multimodal perception of the output, e.g. it notices unprompted when an initial code attempt "solves" with a path around the outside the maze and fixes it
-
Used ChatGPT to Generate a Maze Solution Overlay
By
–
"One try" from me, i.e. I saw the quoted post with the maze, uploaded the image, typed "Solve this maze by adding a red dotted line for the solution path", verified the output in Photoshop, and didn't do any of that twice Share link for the chat: https://
chatgpt.com/share/68014352
-a3ec-800b-a99e-19155a67fa12
… -
Multimodal AI debugging code to solve a maze
By
–
Yes; heavy tool use. It seems to have mostly solved it via code (PIL, cv2), but using multimodal intuition to debug. E.g. one attempt within the CoT generates a path that simply traverses the outside of the maze, but it recognizes on its own this is wrong so it refines the code.
-
Image preprocessing and BFS maze-solving workflow
By
–
The CoT is meandering but it seems to use PIL/cv2 to get the maze into form where it can solve with BFS, notices the resulting solution is broken (simply traverses the outside of the maze) and refines the code until the output looks sane
-
ChatGPT solves a 200×200 maze in one try
By
–
ChatGPT o3 found a path through this 200×200 maze for me in one try. I had to overlay the solution over the original in Photoshop and flip between layers while zoomed in to check the solution never crosses a wall and none of the walls are changed. It's perfect.
-
Cost-per-solution vs cost-per-token for AI models
By
–
I think what Aiden’s saying is that cost per solution makes sense but cost per token does not. If a model A gets the right answer as reliably as model B but at lower cost, the number of tokens it generated to achieve that doesn’t matter.