@localghost

–

01 January 2025 22h38

yep offline data would be really useful

Running DeepSeek on Mobile via Apollo and OpenRouter

By

–

01 January 2025 22h37

all of the models on there are open source but deep seek is too big to run on the phone. you can use Apollo to connect to openrouter to use deepseek though

Running 8B Q4 Model on iPhone 15 Pro RAM

By

–

01 January 2025 22h37

decent results with 8b q4 on a 15 Pro, might be able to squeeze it into a 14 if you’re willing to use most of the phone ram

Inference Performance Not Bad For Extended Usage

By

–

01 January 2025 22h35

it’s not bad, you won’t be constantly inferencing for hours

Smaller AI Models Running on 14-Inch Hardware

By

–

01 January 2025 20h46

some of the smaller models should work on a 14

iOS Hardware Limitations for FP16 Model Execution

By

–

01 January 2025 20h45

this is at q4, don't think it could run fp16 due to iOS limits

Small reasoning models improving offline phone AI capabilities

By

–

01 January 2025 9h06

one of the best you can do offline on a phone for now, but small reasoning models are going to be much better

Ministral 8B Running Offline on Mobile at 10 Tokens/Second

By

–

31 December 2024 20h27

Here’s Ministral 8B running at 10 tok/s on a phone.

Siri isn’t going to save you when you’re stuck without internet. An offline backup AI is essential. pic.twitter.com/nKmjb1Z5jG
— Aaron Ng (@localghost) 31 décembre 2024

Here’s Ministral 8B running at 10 tok/s on a phone. Siri isn’t going to save you when you’re stuck without internet. An offline backup AI is essential.

31 December 2024

Biological vs Machine Feedback Loops and Social Pressure

By

–

31 December 2024 0h49

there was a discussion about biological feedback loops versus machine ones. this was coming out of a priror discussion about it feeling something akin to the social pressure of being polite (the post I made before this one)