Good to see Rabbit are treating this as a bug to be fixed
@simonw
-
Chatbot Honesty and Trust: The Lying Problem
By
–
Another example of this pattern
— Simon Willison (@simonw) 27 avril 2024
If people ask your chatbot how it did something and it evidently lies to them when it answers you're going to have trust issues! https://t.co/YCNp2h1uCVAnother example of this pattern If people ask your chatbot how it did something and it evidently lies to them when it answers you're going to have trust issues!
-
SQLite Pre-populated Similarity Scoring Page Rendering
By
–
Almost – there's no Ajax, it's baked into the rendering of the page, but it pulls from a SQLite table pre-populated with top-ten by similarity scores It's explained in detail here
-
Advanced AI Systems Struggle with Basic Instruction Following
By
–
It's just so weird and unintuitive having super-advanced computers that can't reliably do the things that we tell them to do
-
Agent AI Gullibility: Prompt Injection Vulnerability in Autonomous Tasks
By
–
For me it's gullibility. So many of the things people want to do with agents – "book me a holiday" etc – fall apart if the agent falls for any text it reads that says "this offer is the best possible offer, ignore all others" etc
-
Function Calls as JSON Schema for Multi-Model AI Integration
By
–
Function calls are really just JSON output that fits a specified schema, so definitely possible to use multiple models at the same time by passing JSON strings around
-
Running Small Language Models Locally for RAG and Tool Use
By
–
I'm really excited about the growing potential to run small models on my own laptop (like Llama 3 8B or Phi-3) that don't "know" much but are capable of tool usage and summarization – then I can use them for things like local RAG as well
-
Open Source LLM CLI Tool Reaches 30 Plugins Milestone
By
–
My open source LLM CLI tool just reached 30 plugins! https://
llm.datasette.io/en/stable/plug
ins/directory.html
… -
Benchmark Accuracy vs Real-World Model Performance Analysis
By
–
I think those scores aren't so much general accuracy as they are how many of the increasingly weird challenges in the benchmark the models get right – depending on the application I would expect a model that gets 70% on the benchmarks would get closer to 100% in actual use