My guess is it's not very good at reasoning spatially within ascii representations, or at coming up with better formats. You could try telling it to always look two moves ahead or something. Does it draw boards in its reasoning?
@honnibal
-
Human memory limitations in encoding complex spatial tasks
By
–
Yeah but ask a human to remember the board, or speak the coordinates to them instead of looking at it and see how they do. The encoding of the task is a big confounder to how well the model can reason about it.
-
Avoiding One-Shot Solutions: Multi-Process Testing Approach
By
–
My advice is to not try to get it to one-shot it. Actually I have a pet theory that very stern instruction primes a junior developer role (because that's who gets micromanaged). Just invoke a second process, it's better for the tests not to have the same context anyway
-
Coding Assistants’ Limited View of Correctness and Long-Term Code Quality
By
–
Coding assistants don't get much reward signal for long-term considerations, so they view correctness narrowly. It's easy to wave off style debates and the idea of "anti-patterns", but there's an important idea at the heart of it.
-
Alignment Chart Framework for Codebase Function Classification
By
–
I'm constantly trying out framing devices to get different whole codebase analyses. This one asks to classify functions with an alignment chart (lawful good, chaotic neutral, etc).
-
LLM Reasoning Traces and Bug Detection Capabilities
By
–
Did you look at the reasoning trace? I'm curious whether it did a wide scan and saw that suspicious line, or whether it reasoned from the symptoms to that as a cause. A lot of bugs can be proven incorrect locally, and LLMs are great at that
-
LLM-assisted development risks and prevention strategies
By
–
Mate I'm well aware of that risk, you can read some my thoughts about LLM-assisted development here: https://
honnibal.dev/blog/llm-style
-tips
…. Deleting my home directory would definitely be inconvenient so I'd like to take steps to prevent it. -
Gemini API Payment Methods: GCP Project vs Separate Account
By
–
Basically there's two ways of paying: you can have a Gemini API account, or you can use your GCP project. If you're using GCP you're probably not paying separately for Gemini. In this situation the API key does nothing for you. It doesn't make a lot of sense.
-
Gemini Console API Keys Management and Cross-Team Disagreement Issues
By
–
Sure, but the Gemini console says: "Your API keys are listed below. You can also view and manage your project and API keys in Google Cloud."
I really think teams can't agree on anything so they end up with worst-of-all-worlds solutions. -
Google’s Backwards Incompatible Python SDK Changes Frustrate Developers
By
–
Yeah but this inclination doesn't make it all the way to users in other companies. They deprecated their previous Python SDK, google-generativeai, for a new totally backwards incompatible one called google-genai. It appears Google have no intention of living down the accusations.