CODE

Weights Sharding Strategy: Full Replication Except Last Dimension

By

–

06 October 2024 2h21

Also, tip on weights sharding: as a generic rule of thumb you can use full replication for all variable dimensions except the last one, which should be sharded across the "model" axis

JAX and Keras offer superior large-scale model training performance

By

–

06 October 2024 2h18

In general, you will find that JAX (and in particular JAX + Keras for the super simple API) offers an exceptionally smooth and performant large-scale model training experience. Zero hassle and high utilization out of the box — there is no alternative that comes close to it.

ModelParallel Scaling: Multi-Host Training Setup and Seeding Requirements

By

–

06 October 2024 2h17

The ModelParallel scheme above scales to arbitrary model sizes and numbers of devices on a single host, but it also works with multi-host training. If you do multi-host training, a common gotcha is that you need to remember to seed everything in the same way on each host (else

Keras JAX Data and Model Parallelism Configuration

By

–

06 October 2024 2h12

1. Configuring data parallelism for Keras models running on JAX (works with fit/evaluate/predict with 0 other changes to your code). All model state and incoming dataset are automatically distributed. 2. Configuring model parallelism + data parallelism while specifying the

→ View original post on X — @miketamir

Augmenting Statistical Models with Natural Language Parameters

By

@miketamir

–

05 October 2024 21h01

Augmenting Statistical Models with Natural Language Parameters https://
bit.ly/3XS5DmX
#AI #MachineLearning #DeepLearning #LLMs #DataScience

→ View original post on X — @akshay_pachaar

gptme: Open-Source AI Agent Tool for Your Terminal

By

@akshay_pachaar

–

05 October 2024 18h39

Your terminal just got smarter! Introducing gptme: an open-source tool that let's you run AI Agent in your terminal with local tools Here's why it's a game-changer: Free and open-source Runs code in your local environment Reads, writes, and edits files Browses

→ View original post on X — @avikumart_

Master NumPy Indexing for Machine Learning Development

By

@avikumart_

–

05 October 2024 17h00

Great way to learn numpy indexing

→ Voir le post original sur X — @flashtweet

India’s $1.2B AI Push: Gemini Live and ChatGPT Code Advances

By

@flashtweet

–

05 October 2024 16h32

ℕ𝕒𝕞𝕒𝕤𝕥é 𝕀𝕟𝕕𝕀𝔸
L’Inde veut devenir un acteur majeur de la révolution de l'intelligence artificielle. Elle a déjà débloqué 1,2 milliards de dollars pour y parvenir.
Google lance son assistant vocal Gemini Live AI en hindi
Chat GPT fait une offensive dans le code