An anecdote for you from LLM land: I sent Opus 4.7 out to do a literature survey of various government law enforcement internal guidelines w/r/t AML and KYC usage in prosecutions. It was much more successful at finding these docs than I would have expected. I then told it:
@patio11
-
LLM Detects Marketing Patterns Even in Truncated Text Excerpts
By
–
Bingoes a 72 word excerpt about marketing strategy with no firm-specific information included at all, citing a stylistic device. Truncated to 49 words to exclude that device; still bingoes it. Truncated to 8 words; somewhat huffily refuses to speculate.
-
AI Refusals Mask Truth With False Self-Deprecation
By
–
Several more refusals in that genre, and this salaryman is left with the distinct impression that his counterparty has considered telling him the truth, come to the conclusion that that is undesirable, and has substituted (false) self-deprecation regarding one's capabilities.
-
AI Correctly Identifies Redacted Federal Agency Communication From 15 Years Ago
By
–
Correctly bingoes a 3 paragraph communication to a federal agency written 15 years ago where all biographical hints were replaced with [REDACTED FOR PURPOSE OF EVAL].
-
Model Weights Alone Reveal Identity Without System Prompts
By
–
To head off the obvious question: This was conducted via the API and so it should not have hidden memory/system prompts/etc which leak the identity of yours truly to the model. The model weights, alone, got it to enough information.
-
Claude Opus Identifies Writer by Stylistic and Biographical Tells
By
–
Opus' rationale for the one it got right begins with the accurate and somewhat disconcerting "RATIONALE: Several strong stylistic and biographical tells point to Patrick McKenzie."
-
Claude Opus 4.7 Reproduces Writing Samples in Informal Test
By
–
FYI, I casually tried to reproduce this on three writing samples from many years ago, which I do not believe to be on the public Internet and which were varying subjective difficulty levels. Opus 4.7 went 1 for 3 with ~no effort in the prompt (~two sentences long).
-
ChatGPT Used as Legal Drafting Assistant for Contract Clauses
By
–
ChatGPT esquire also flagged the consideration issue and suggested explicit forbearance of soda on non-enumerated occasions in preference to the above-suggested recital. Ah, I love modernity.
-
Use Agentic Coding UX Over Chat for Best Results
By
–
A recommendation which I have not tested for this exact task but which I feel moderately strongly about due to a closely related task: if you're familiar with Claude Code, Codex, Cowork, or other of the related UXes, use that in preference to regular chat window.
-
Using LLMs to Review Tax Returns Before Signing
By
–
Happy almost tax day. FYI: The most dollar-denominated impactful thing I've ever done for geeks, by sum of claims in my inbox, is the piece on salary negotiation. The second most impactful, rising fast, is "Have an LLM review your taxes before you sign them."