CODE

→ View original post on X — @perplexity_ai

Fast REST API for Open-Source LLMs with Lower Latency

By

@perplexity_ai

–

04 October 2023 22h21

Key features: • Easy-to-use REST API • Up to 2.9x lower latency than Replicate & 3.1x lower than Anyscale • Reliable, battle-tested infra, serving 1B tokens in our prod environment daily • One stop shop for open-source LLMs Read the API specs:

→ View original post on X — @perplexity_ai

Open-source LLM API enables fast inference without GPU requirements

By

@perplexity_ai

–

04 October 2023 22h21

Ideal for developers seeking to incorporate open-source LLMs into their products, our API offers fast inference speeds without requiring extensive C++/CUDA knowledge or GPU access. Subscribe to Perplexity Pro to try it out: http://
pplx.ai/pro

→ View original post on X — @googleai

JAX Library Enables Computer Vision on Spherical Surfaces

By

@googleai

–

04 October 2023 20h49

Applying computer vision models designed for planar images to data projected on spherical surfaces is challenging. Here we present an open-source library in JAX to solve the challenges of rotation and regular sampling for state-of-the-art performance → https://t.co/wXdIpkmtDy pic.twitter.com/0mC7PdeLW4
— Google AI (@GoogleAI) 4 octobre 2023

Applying computer vision models designed for planar images to data projected on spherical surfaces is challenging. Here we present an open-source library in JAX to solve the challenges of rotation and regular sampling for state-of-the-art performance → https://
goo.gle/46z3vD7

→ View original post on X — @officiallogank

OpenAI Python SDK 1.0 Release: Beta Testing Now Available

By

@officiallogank

–

04 October 2023 20h33

Exciting news for @OpenAI devs: we are close to a 1.0 release of the OpenAI Python SDK . You can test the beta version of 1.0 today, we would love to get your early feedback!

PEFT LoRA Implementation Guide for Model Fine-tuning

By

–

04 October 2023 18h27

Just paste it directly above your call to get_peft_model(). There's many examples around, eg: https://
github.com/huggingface/pe
ft/blob/main/examples/conditional_generation/peft_lora_seq2seq.ipynb
…

BNB Quantized Layers Initialization Performance Issues

By

–

04 October 2023 18h25

The issue is the creation of the bnb quantised layers, not the adapters. (They are fast to init anyway since they're small.)

WhatsApp’s Engineering Challenges: Lessons Beyond ML Data Engineering

By

–

04 October 2023 18h24

Consider what Whatsapp achieved prior to acquisition with their team. And their eng challenges were many orders of magnitude harder than ML data eng!

RLHF Training with Small Teams and Data Engineering

By

–

04 October 2023 18h23

I've seen plenty of folks doing RLHF with <5 dedicated people! (And I do all my own data engineering — it doesn't take too long after a few decades of practice…)