OPEN SOURCE

–

25 June 2024 4h51

Which LLM/model? Can I choose from any opensource models? Does this use Ollama or LM Studio for OpenAI compatible API at the backend?

→ View original post on X — @karpathy

25 June 2024

llm.c Memory Efficiency Advantages Over PyTorch

By

@karpathy

–

24 June 2024 21h56

If you match the parameters you're actually under-estimating the improvement, because llm.c takes up a lot less space so you can crank up the batch size. I haven't fully dug into why PyTorch takes up that much space, slightly worried I'm doing something wrong but not sure what

→ View original post on X — @karpathy

MLX Framework as Optimized Alternative to PyTorch MPS

By

@karpathy

–

24 June 2024 20h58

I'd have a look at MLX, cc @awnihannun , from some recent chatter I understand it is a lot more optimized that PyTorch mps atm. This would need a re-write of the build-nanogpt code into mlx, but possibly it's not that involved. I'd be happy to accept a PR for mlx clone.

ChatGPT Artifacts Implementation Already Available on GitHub

By

–

24 June 2024 20h21

This guy already did it here: https://
github.com/ozgrozer/chatg
pt-artifacts/tree/main
…

Running Llama-3 8B with 16GB RAM Using Quantization

By

–

24 June 2024 20h04

Just an FYI — you can run Llama-3 8B with 16GB of RAM. And there are even quantized versions which lets you run it on even less memory.

Run Llama-3 Locally with Ollama or LM Studio

By

–

24 June 2024 18h09

You can do it either by using @ollama or @LMStudioAI
. Have them spin up Llama-3 endpoint locally in OpenAI format, plug that with the frontend created with Calude and you are good to go!!

→ View original post on X — @sumanth_077

Install Clarifai SDK Llama-index Setup Guide Instructions

By

@sumanth_077

–

24 June 2024 16h36

Step 1: Install the Clarifai Python SDK, Llama-index, and set your Clarifai Personal Access Token as an environment variable. Sign up for the platform, and you can access the PAT here: https://
clarifai.com/settings/secur
ity
…