AI - AI Dynamics

Simple Prompting Techniques Achieve 49 F1 Score Baseline

By

–

09 August 2023 21h33

2/ We started with simple prompting — tell the model what you're looking for and how to format its responses. This catches a lot of the low hanging fruit (~49 F1), but still needs refinement.

Data-Centric Prompt Engineering: Iterative Model Error Refinement

By

–

09 August 2023 21h33

3/ We then began data-centric prompt engineering, iteratively identifying model error modes and addressing them in the prompt where possible, either with added detail in the prompt or adding in-context examples (~68 F1).

DistilBERT Fine-Tuning Achieves 84 F1 With Programmatic Labeling

By

–

09 August 2023 21h33

4/ For maximum quality, we finished with fine-tuning, using Snorkel Flow to programmatically label data in areas where the model was still underperforming. Our best model (~84 F1) is fine-tuned DistilBERT, a model 10,000x smaller than GPT-4.

Foundation Models: Data-Centric Techniques for Business Quality

By

–

09 August 2023 21h33

5/ Foundation models are great, but they're just a foundation. For business-critical applications where quality matters, use the full slate of data-centric model improvement techniques!

Fine-tuning and Prompt Engineering Outperform GPT-4 Baseline

By

–

09 August 2023 21h33

1/ Prompt engineering LLMs is valuable, but used alone leaves many points on the table. See how we combine prompt engineering and fine-tuning to create a specialized model that outperforms prompted GPT-4 by 15 to 34 points on a real-world extraction task! https://
snkl.ai/pro