My personal interest this year is the interaction of LLMs and small SaaS businesses. I suspect they (we?) are early big winners of the industrial revolution we’re presently going through.
@patio11
-
Using an LLM for real-world information retrieval and task assistance
By
–
So many little wins, too. The insurance company asks for six fields about my primary care physician. Me to LLM: [uncommon name], Chicago, [institutional affiliation]. Google please.
LLM: Do you go see her at Main Street or Example Street?
Me: Example Street.
LLM: *gets it done* -
AI models performing real-world professional tasks
By
–
Also note that ex-chatbot these things can do Real Work (TM) which doesn't look like systems engineering and you can check that Real Work (TM) in the typical ways that one checks Real Work (TM). They can compile a balance sheet from source documents.
-
Expert adoption and risk management of LLMs in professional domains
By
–
I think better calibrated experts use LLMs in e.g. adjacencies to their professional domain fairly pervasively these days, accepting some risk of a hallucination based on e.g. cost of checking evidentiary trail, how important that output is and where it is exposed, etc.
-
The challenge of verifying LLM output accuracy
By
–
Even accounting for hallucinations, there is substantial overlap between what LLMs are *manifestly* capable of doing and what a given user is *not conveniently capable of checking*, including when a relatively sophisticated user would say "I mean just check source URLs."
-
Reflecting on the challenges of LLM alignment through analogies
By
–
"There is substantial overlap between the smartest bears and the least-equipped humans" was once an observation on how it is difficult to engineer a bear-proof garbage receptacle for national parks. I think a spiritually similar thing happens with LLMs.
-
Using LLMs for Automated Bug Generalization and Code Analysis
By
–
As one non-limiting example, a sufficiently motivated engineer could, as a hobby project, build a pipeline to reverse engineer post-Mythos patches then describe the bugs found in markdown and tell a less-creative LLM “Generalize from this data set please against this code base.”
-
Mitigating AI Hallucinations via Inference Redundancy and Voting
By
–
95% reliable is ~100% reliable if the hallucinations are uncorrelated across runs/systems/etc. It just means you spend a bit of extra effort on implementing a voting/etc layer and spend more on inference redundantly. This is almost boring.
-
Analyzing the nature of AI hallucinations for safety and reliability
By
–
A related manner: there are many people who are quite worried that hallucinations will make AI unsafe or inappropriate for broad classes of problems. I very rarely hear those people explicitly try to reason whether hallucinations are a) random or b) persistent given request.
-
The gap between technologists and society on LLM capabilities
By
–
“And if you think that’s impressive, wait until someone puts it in a for loop.” a persistent source of why technologists and civil society have different POVs of the impact of various LLM capabilities improvements.