Yeah, agree that it's a hard problem. It might be the EQ version of uncanny valley.
@karpathy
-
AI manipulation through personalized algorithms raises ethical concerns
By
–
yes exactly! a bit like i'm being manipulated in some creepy way. "please like me, look how much i know about you, we are good friends".
-
Memory Systems and RAG Limitations in AI Models
By
–
If I had to guess it's less decay and more that memories have naive RAG-like implementations, so you're at the mercy of whatever happens to retrieve in the top k via embeddings. They don't process you in aggregate and over time (probably compute constraints) so they struggle to
-
LLMs Overfitting to RAG Context: A Systemic Training Bias
By
–
(I cycle through all LLMs over time and all of them seem to do this so it's not any particular implementation but something deeper, e.g. maybe during training, a lot of the information in the context window is relevant to the task, so the LLMs develop a bias to use what is given, then at test time overfit to anything that happens to RAG its way there via a memory feature (?))
-
The Challenges of Personalization in Language Models
By
–
One common issue with personalization in all LLMs is how distracting memory seems to be for the models. A single question from 2 months ago about some topic can keep coming up as some kind of a deep interest of mine with undue mentions in perpetuity. Some kind of trying too hard. [Translated from EN to English]
-
PyPI Security Incident: 425K Downloads Exposure Timeline
By
–
In particular this clarifies the timeline more: 1.82.7 was published 10:39 UTC, PyPI quarantine approx 13:38, so this was up ~3 hours. At 3.4M downloads/day this might be approx ~425K downloads, a lot of that could be non-latest/locked versions so maybe 20K – 80K range exposure.
-
LiteLLM PyPI Supply Chain Attack Steals Credentials and API Keys
By
–
Software horror: litellm PyPI supply chain attack. Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords. LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm. Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery – Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks. Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages. Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible. Daniel Hnyk (@hnykda) LiteLLM HAS BEEN COMPROMISED, DO NOT UPDATE. We just discovered that LiteLLM pypi release 1.82.8. It has been compromised, it contains litellm_init.pth with base64 encoded instructions to send all the credentials it can find to remote server + self-replicate. link below — https://nitter.net/hnykda/status/2036414330267193815#m
-
Context and Specification Engineering Beyond Prompting Techniques
By
–
Yes I think in one part of the video I think I still used the word prompt but it's not really about "prompting", it's about context and spec engineering, and then all of the other harness things – tools, workflows, etc.
-
AI Agents Generate Poor Code Quality and Ignore Instructions
By
–
I'm not very happy with the code quality and I think agents bloat abstractions, have poor code aesthetics, are very prone to copy pasting code blocks and it's a mess, but at this point I stopped fighting it too hard and just moved on. The agents do not listen to my instructions
-
AI as Teammate: Building Partnership with Intelligent Assistants
By
–
Great questions! Starting backwards with (3), I'd hope AIs can feel like Rocky from Project Hail Mary (it's top of mind having seen it yesterday), like a partner and a teammate. As one small example that stuck with me recently, when Claude found the Sonos system on my LAN, it