Commit Graph

9 Commits

Author SHA1 Message Date
Eren Gölge 5911eec3b1 Small trainer refactoring
1. Use a single Gradscaler for all the optimizers
2. Save terminal logs to a file. In DDP mode, each worker creates `trainer_N_log.txt`.
3. Fixes to allow only the main worker (rank==0) writing to Tensorboard
4. Pass parameters owned by the target optimizer to the grad_clip_norm
2021-08-26 17:08:58 +00:00
Eren Gölge ecf5f17dca Fix distribute.py and ddp training 2021-08-12 22:22:32 +00:00
Eren Gölge b3c073c99b Allow runing full path scripts with `distribute.py` 2021-06-28 17:03:47 +02:00
Eren Gölge a7617d8ab6 Add 🐍 python 3.9 to CI 2021-06-28 17:03:47 +02:00
Eren Gölge 45947acb60 Update `TTS.bin` scripts for the new API 2021-06-28 17:03:47 +02:00
Eren Gölge 03494ad642 adjust `distribute.py` for the `train_tts.py` 2021-06-28 17:03:19 +02:00
Eren Gölge d0ab0382fc linter fixes 2021-06-01 09:15:32 +02:00
Eren Gölge f519012dea reformatting and styling 2021-04-12 11:47:39 +02:00
erogol c5074cfd8e general purpose distribute.py 2020-10-08 01:30:42 +02:00