The recording from our Flow Engineering webinar with @hwchase17 and @itamar_mar is up! https://
youtube.com/watch?v=eBjxz7
qrNBs
… "Flow Engineering" is a term that has been gaining in popularity recently. The first time it was mentioned as term was in CodiumAI paper on AlphaCodium, where they
RESEARCH
-
Flow Engineering Webinar Recording with LangChain Leaders
By
–
-
Kernel optimization attempts yield minimal performance gains
By
–
added under kernel4 https://
github.com/karpathy/llm.c
/commit/cb791c4ef58d45d58e5af624b0ed41439ac7aeff
…
a bit surprised to only see ~1-2% out of it, which then washes out in training, as the layernorm is not a top-ranking time kernel. Also tried float4 and unrolling but that didn't improve it too much bleh -
PyTorch Pay-Per-Use Model Through Advanced Compiler Technology
By
–
I think some day, projects like PyTorch will make users pay only for the overhead they use. Lots of R&D needed to get there: compiler technology, type systems, various other things. Actually, LuaJIT was really cool in that aspect — one of the most brilliant pieces of
-
AI Pioneer Daniel Dennett Remembered for Consciousness Research
By
–
The AI world mourns the loss of Daniel Dennett, a true pioneer who illuminated the complexities of consciousness and thought. His visionary ideas will forever inspire and challenge us. We stand on the shoulders of a giant. Rest in peace, Daniel.
-
Major Data and AI Event: GenAI and Engineering Conference June 10-13
By
–
POV: you would do anything to get into the biggest data and #AI event of the year.
— Databricks (@databricks) 19 avril 2024
Register to hear from leading experts, researchers, and open source contributors about topics like #GenAI and data engineering and streaming.
Join us June 10-13: https://t.co/Rl8WajxnNn pic.twitter.com/x2fNrVOgAJPOV: you would do anything to get into the biggest data and #AI event of the year. Register to hear from leading experts, researchers, and open source contributors about topics like #GenAI and data engineering and streaming. Join us June 10-13: https://
bit.ly/3UlaDA7 -
Lava Lamps: The Random Number Generators Securing Internet
By
–
Randomness is hard to achieve. It is why the security of 10% of the internet is secured by a wall of lava lamps watched by a camera to generate true randomness
-
Death of Daniel Dennett: His Prophetic Warnings on AI
By
–
Very sad to hear that Daniel Dennett passed away. I interviewed him back in 2017, after the publication of his latest book, and he offered a very prescient warning about AI systems that might seem incredibly clever but work in mysterious ways: https://technologyreview.com/2017/04/11/5113/the-dark-secret-at-the-heart-of-ai/ …
-
GPT-2 as the Hello World of Large Language Models
By
–
GPT-2 is the "hello world" of LLMs I think (there must be a better analogy… err MOS 6502? xv6?), so that's why I started there. And it has a proper paper, weights released and available, and a lot is known about it. At this point it is an artifact of historical significance.
-
Lack of Public Documentation in AI Implementation Practices
By
–
That sounds like it would work in theory, I'd love to see documented examples with live demos of people doing that in practice I get the impression that people who do this rarely share their results publicly, which is a shame
-
Synthesizing AI Responses with RAG Pipelines and SFT Models
By
–
It can be done: Gather any available real query examples, add in synthetic Q’s derived from docs. Synthesize responses using RAG pipeline. SFT (a smaller model) on resulting Q&A’s, without RAG context. Whether it saves any money is another question.