OPEN SOURCE

→ View original post on X — @perplexity_ai

Fast REST API for Open-Source LLMs with Lower Latency

By

@perplexity_ai

–

04 October 2023 22h21

Key features: • Easy-to-use REST API • Up to 2.9x lower latency than Replicate & 3.1x lower than Anyscale • Reliable, battle-tested infra, serving 1B tokens in our prod environment daily • One stop shop for open-source LLMs Read the API specs:

→ View original post on X — @perplexity_ai

Open-source LLM API enables fast inference without GPU requirements

By

@perplexity_ai

–

04 October 2023 22h21

Ideal for developers seeking to incorporate open-source LLMs into their products, our API offers fast inference speeds without requiring extensive C++/CUDA knowledge or GPU access. Subscribe to Perplexity Pro to try it out: http://
pplx.ai/pro

→ View original post on X — @googleai

JAX Library Enables Computer Vision on Spherical Surfaces

By

@googleai

–

04 October 2023 20h49

Applying computer vision models designed for planar images to data projected on spherical surfaces is challenging. Here we present an open-source library in JAX to solve the challenges of rotation and regular sampling for state-of-the-art performance → https://t.co/wXdIpkmtDy pic.twitter.com/0mC7PdeLW4
— Google AI (@GoogleAI) 4 octobre 2023

Applying computer vision models designed for planar images to data projected on spherical surfaces is challenging. Here we present an open-source library in JAX to solve the challenges of rotation and regular sampling for state-of-the-art performance → https://
goo.gle/46z3vD7

PEFT LoRA Implementation Guide for Model Fine-tuning

By

–

04 October 2023 18h27

Just paste it directly above your call to get_peft_model(). There's many examples around, eg: https://
github.com/huggingface/pe
ft/blob/main/examples/conditional_generation/peft_lora_seq2seq.ipynb
…

Hugging Face Performance Fix Coming Soon Official Update

By

–

04 October 2023 18h08

I chatted to the Hugging Face folks and they tell me an official fix for this performance issue is on the way! 😀 So hopefully my hacky workaround will be redundant soon…

PEFT Kaiming Init Optimization Patch for Training

By

–

04 October 2023 18h08

The issue is that PEFT spends all that time doing unnecessarily doing kaiming init. So our code first patches the `init` module so that `kaiming_uniform_` does nothing at all. Here's a gist with the full code:

Speed up Hugging Face PEFT model initialization with this trick

By

–

04 October 2023 18h08

If you use @huggingface PEFT, then you'll have noticed that calling `get_peft_model` is *really* slow… However, you can make it nearly instant by running this magic snippet first!

Small Team Creates Best 7B Model in Three Months

By

–

04 October 2023 17h56

It's plenty enough. They created the best 7B model yet with a small team in 3 months. The size of an eng time is not generally positively correlated with its productivity.

→ View original post on X — @hardmaru

Fast SDXL Inference with JAX on TPU v5e

By

@hardmaru

–

04 October 2023 15h51

Fast SDXL Inference with JAX on TPU v5e Amazing how fast anyone can generate 4 1024×1024 resolution SDXL images in under 5 seconds, for free: https://
huggingface.co/spaces/google/
sdxl
…