In my experience those results were quite suboptimal and didn’t quite result in usable transcriptions. With better decoding strategy those issues can be alleviated a bit. So the hack is more so of using contrastive search with Whisper to enable such use cases. Will run more
@reach_vb
-
Fine-Tuning Whisper Model for Improved Performance
By
–
Definitely yes! You can fine-tune Whisper to boost the model performance: https://
huggingface.co/blog/fine-tune
-whisper
… -
Contrastive Search Outperforms Greedy Search in Text Generation
By
–
Interesting! Can you try running the next cell with contrastive search? In my experience greedy search doesn’t perform as well! Would be curious to see the results 🙂
-
AI Transcription Accuracy and Generation Strategy Optimization
By
–
Yess! Quite possible – although the accuracy of the transcriptions still needs to be checked. I’ve found it to loose context if not used with the right generation strategy!
-
Contrastive Search Benchmarks Coming Soon for Developers
By
–
Definitely! More benchmarks on contrastive search coming soon too Thanks for empowering millions of developers
-
Contrastive Search vs Greedy Search for LLM Generation
By
–
Yeh! Would deffo recommend contrastive search. It works quite well (however, over generates sometime ) Greedy search simply results in lost context.
-
Fine-tuned Models Performance on Multilingual Translation Tasks
By
–
Agreed! From my experience, it works well on languages that were abundant in the train set and also had lang -> en translate pairs. I’ll run some experiments to see how well do fine-tuned models perform. Quite lovely to see ya already using this
-
Whisper Transcription Capabilities and Language Translation Features
By
–
Ha! Semantics – in this case we literally force it to transcribe into a target language. So, technically you can say it is translating! Whisper traditionally only translates from language “X” into English.
-
AI Video Generation from Audio Using Word-Level Timestamps
By
–
The audio snippet was taken from "Stupid People" comedy routine by Bill Engvall The video was generated via @mhollemans
's brilliant Word-level timestamps space: -
Testing Model Performance on Untrained Tasks: Benchmarking Results
By
–
Quick caveat: It's a task that the model isn't explicitly trained on, so test it thoroughly before us We'll run more benchmarks this week and share our findings soon! Help us test it better? 🙂