Very sad to hear that Daniel Dennett passed away. I interviewed him back in 2017, after the publication of his latest book, and he offered a very prescient warning about AI systems that might seem incredibly clever but work in mysterious ways: https://technologyreview.com/2017/04/11/5113/the-dark-secret-at-the-heart-of-ai/ …
AI
-
NVDA PE Ratio Returns to November 2022 Levels Despite Higher Stock Price
By
–
NVDA now has a PE ratio comparable to what it was in November 2022, when it was trading around $150-160 per share.
-
GPT-2 as the Hello World of Large Language Models
By
–
GPT-2 is the "hello world" of LLMs I think (there must be a better analogy… err MOS 6502? xv6?), so that's why I started there. And it has a proper paper, weights released and available, and a lot is known about it. At this point it is an artifact of historical significance.
-
GPT-2 Weight Initialization and Fine-tuning Strategy
By
–
atm we're doing init from gpt-2 weights and finetuning. this was very useful for debugging and when the code was slower. there is no code yet to init from scratch, so no code to warmup the lr etc. should be a very short addition though.
-
Lack of Public Documentation in AI Implementation Practices
By
–
That sounds like it would work in theory, I'd love to see documented examples with live demos of people doing that in practice I get the impression that people who do this rarely share their results publicly, which is a shame
-
Synthesizing AI Responses with RAG Pipelines and SFT Models
By
–
It can be done: Gather any available real query examples, add in synthetic Q’s derived from docs. Synthesize responses using RAG pipeline. SFT (a smaller model) on resulting Q&A’s, without RAG context. Whether it saves any money is another question.
-
Meta AI vs ChatGPT: Internet Search Capabilities Compared
By
–
That's not an equal comparison, because Meta AI has the ability to search the internet and incorporate the search results into its responses – ChatGPT can do that too with Bing and GPT-4, but can't do that with 3.5
-
Waiting for New Model Evaluation Results
By
–
nothing gets my heart rate up like waiting for eval results on new models to come in
-
Fine-tuning GPT-3.5: Use Cases and Community Experiences
By
–
Using https://
platform.openai.com/docs/guides/fi
ne-tuning
… ? What have you used that for – I've not heard many examples of people fine-tuning 3.5 -
Team motivation and trust are fundamental regardless of pressure
By
–
if people aren't motivated and you can't trust them you are anyway lost in any team no matter pressure or not
