MACHINE LEARNING

→ View original post on X — @skirano

Reddit Instruct Dataset Released for AI Research

By

@skirano

–

08 February 2024 2h51

Huge thanks to @teknium for their work and to @Euclaise_ for providing the dataset! https://
huggingface.co/datasets/eucla
ise/reddit-instruct-curated
…

→ View original post on X — @skirano

Local Model Fine-tuning Tutorial and Data Formatting Guide

By

@skirano

–

08 February 2024 2h51

If you are interested in learning more about fine-tuning models locally, consider following me. I will post a tutorial on how I did this fine-tuning, how I formatted the data, etc.

PyTorch torch compile encountered critical error during execution

By

–

08 February 2024 2h06

welp. this is what happened when i tried to use torch compile

Parallel Batch Processing Optimization in Machine Learning

By

–

08 February 2024 0h01

what would that even mean? why would you not just combine “parallel” batches into a bigger batch?

Weight Decay Impact on Model Training Speed

By

–

07 February 2024 21h46

what does increasing weight decay do in this case? I’d expect that to slow training down

Autograd Backward Pass Implementation Progressing Successfully

By

–

07 February 2024 19h32

and here's a slightly less-zoomed backward pass. autograd is running, backward is happing, etc. etc. seems all fine and good to me

GPU utilization troubleshooting: why only 30% despite full memory

By

–

07 February 2024 19h32

yet here's proof that my GPU utilization is still trash. if forward or backward is ~always happening according to the profiler, and memory is decently full now (at least over 50%)… …then how is my GPU being used only on average 30% of the time? what am i missing here?! help

PyTorch GPU Utilization Issues: Profiling Forward Backward Pass

By

–

07 February 2024 19h32

alright y'all. still having GPU utilization issues and against my better judgement i broke out the PyTorch profiler. posting a few screenshots. doesn't seem like I/O is an issue (no gaps) how is GPU util so bad (~30%) if either forward or backward is running almost always?