In my mind, “decide the control flow” = “if statements based on the outcome of an LLM” Agree it’s a tough definition
LLMS
-
Token Caching Challenges in Large Language Models
By
–
Not a dumb question at all. I think caching is the trickiest one here (obvious ones like kv-caching aside). Caching token embeddings of common words probably don’t really help much. And prompts are probably often diverse enough that caching those would be too expensive. Session
-
Serving ChatGPT to 100M users with 1k H100s efficiently
By
–
Load balancing, caching, dynamic batching… with 10% DAU you probably don’t need much more than 1k H100s to serve ChatGPT to 100 million customers.
-
Memory Implementation in AI Systems: Few-shot vs Prompt Updating
By
–
Super interesting. What does memory look like? Few shot examples or updating prompt?
-
From Bench to Bot: The Importance of Prompt Engineering
By
–
From bench to bot: How important is prompt engineering?
#AI #AIio #BigData #ML #NLU #Futureofwork @gp_pulipaka @stratorob @PetiotEric @EvanKirstel @Fgraillot @HaroldSinnott @HeinzVHoenen @helene_wpli -
Real Inference Requires Optimization and Search
By
–
In my opinion, *real* inference involves some form of optimization/search.
How about "real inference" ? -
Optimizing llama.cpp quantization performance gains
By
–
good reminder: I need to check my llama.cpp quants I suspect I’m leaving perf on the table.
-
Sakana AI Hiring Applied Engineers for AI Implementation
By
–
Sakana AIはグローバルで最先端のAI技術を日本の課題解決のために実装したいと考えています。本日、AI技術をお客様の現場に導入する業務を担うApplied Engineerを募集します。LLMやAIエージェントを用いて一緒にビジネス価値を作りたい方は、ぜひご応募ください。 https://
sakana.ai/careers/ -
Llava o1: Open-Source Vision Language Model with CoT
By
–
Llava o1: https://
huggingface.co/Xkev/Llama-3.2
V-11B-cot
… -
LLMs Set for Major Speed and Cost Improvements Ahead
By
–
I’d bet on orders of magnitudes of speed and cost improvements to come for LLMs. Future is bright