WeberJulian
6b03943526
Move multilingual logic out of the trainer
2021-12-20 11:54:10 +00:00
Edresson
2b2cecaea2
Set the new_fields in copy_model_files as None by default
2021-12-20 11:54:10 +00:00
WeberJulian
1472b6df49
make style
2021-12-20 11:54:10 +00:00
WeberJulian
3b5592abcf
fix test vits
2021-12-20 11:54:10 +00:00
Edresson
0bdfd3cb50
Add the ValueError in the restore checkpoint exception to avoid problems with the optimizer restauration when new keys are addition
2021-12-20 11:54:09 +00:00
Edresson
f996afedb0
Implement multilingual dataloader support
2021-12-20 11:54:09 +00:00
Eren Gölge
512ada7548
Fix callbacks against multi-gpu training
2021-12-01 10:32:14 +00:00
Eren Gölge
70e4d0e524
Fix grad_norm handling
2021-10-21 16:29:06 +00:00
Eren Gölge
588da1a24e
Simplify grad_norm handling in trainer
2021-10-19 16:33:04 +00:00
Eren Gölge
b4b890df03
Update trainer's initialization
2021-10-18 08:53:19 +00:00
Eren Gölge
073a2d2eb0
Refactor VITS multi-speaker initialization
2021-10-15 10:20:00 +00:00
Eren Gölge
e15bc157d8
Fix #873
2021-10-14 14:39:45 +00:00
Eren Gölge
37959ad0c7
Make linter
2021-09-30 23:02:16 +00:00
Eren Gölge
55d9209221
Remote STT tokenizer
2021-09-30 14:58:26 +00:00
Eren Gölge
8ada870a57
Refactor `trainer.py` for v2
2021-09-30 14:16:34 +00:00
Eren Gölge
a97dc8d09f
Fix trainer malformatted print
2021-09-13 08:32:02 +00:00
Eren Gölge
66732025e1
Add `base_model` field to `forward_tts` configs
2021-09-10 17:23:48 +00:00
Eren Gölge
2dfc5bdd11
Fix best_model_path init if no best_mode
2021-09-09 09:01:52 +00:00
Eren Gölge
abf5e48177
Fix logging current learning rate in trainer
2021-09-09 09:01:04 +00:00
Eren Gölge
6c4c1065b0
Fix trainer's scheduler restoring
2021-09-09 09:00:27 +00:00
Eren Gölge
e20ea57c87
Update comment and add a warning
2021-09-07 12:23:32 +00:00
Eren Gölge
91a70e80b2
Refactor TTSDataset
...
Return a dict by `collate`
Refactor batch handling in `collate`
A couple of bug fixes
2021-09-06 15:16:58 +00:00
Eren Gölge
8584f2b82d
Update docstring format
2021-09-06 15:16:58 +00:00
Eren Gölge
9af42f7886
Restore `last_epoch` of the scheduler
2021-09-06 15:16:58 +00:00
Eren Gölge
738eee0cf9
Fix style
2021-08-30 13:12:13 +00:00
Eren Gölge
c560114324
Fix #750
2021-08-30 13:06:50 +00:00
Eren Gölge
18da8f5dbd
Update pylint 2.10.2 and fix lint issues
2021-08-30 08:10:35 +00:00
Eren Gölge
f186856e5d
Add option to sort input sequnce by audio len
2021-08-30 08:10:35 +00:00
Eren Gölge
2620f62ea8
Move duration_loss inside VitsGeneratorLoss
2021-08-27 07:07:07 +00:00
Eren Gölge
5911eec3b1
Small trainer refactoring
...
1. Use a single Gradscaler for all the optimizers
2. Save terminal logs to a file. In DDP mode, each worker creates `trainer_N_log.txt`.
3. Fixes to allow only the main worker (rank==0) writing to Tensorboard
4. Pass parameters owned by the target optimizer to the grad_clip_norm
2021-08-26 17:08:58 +00:00
Eren Gölge
c5d1dd9d1b
Fix restoring best_loss
...
Keep the default value if model checkpoint has no `model_loss`
2021-08-17 12:12:36 +00:00
Eren Gölge
c8bbcdfd07
Fix `test_run` for DDP
2021-08-13 19:39:02 +00:00
Eren Gölge
7c0d564965
Syncronize DDP processes
2021-08-13 10:40:50 +00:00
Eren Gölge
ecf5f17dca
Fix distribute.py and ddp training
2021-08-12 22:22:32 +00:00
Eren Gölge
39004484b9
Fix 🐛
...
Fix synthesizer multi-speaker init
Fix #712
2021-08-10 12:56:32 +00:00
Ayush Chaurasia
e685ddfca7
Update trainer.py
2021-08-09 18:37:46 +00:00
Ayush Chaurasia
8a246cbb66
Update trainer.py
2021-08-09 18:35:08 +00:00
Ayush Chaurasia
f3e9d61330
Refactor logging initialization
2021-08-09 18:35:08 +00:00
Ayush Chaurasia
79b74a989d
Update: add_text
2021-08-09 18:34:38 +00:00
Ayush Chaurasia
936a47504d
Update Logger API, recipes
2021-08-09 18:34:00 +00:00
Ayush Chaurasia
f63cf46c55
Unified logger API
2021-08-09 18:34:00 +00:00
Ayush Chaurasia
f4434da5a3
Update disabled structure
2021-08-09 18:31:16 +00:00
Ayush Chaurasia
f606741dc4
Add artifacts logging , wandb args
2021-08-09 18:31:16 +00:00
Ayush Chaurasia
f5e50ad502
WandbLogger
2021-08-09 18:27:06 +00:00
Eren Gölge
bf562cf437
Update `trainer.py`
...
Fix multi-speaker initialization of models. Add changes for end2end`tts`
models.
2021-08-09 18:02:36 +00:00
Agrin Hilmkil
ced4cfdbbf
Allow saving / loading checkpoints from cloud paths ( #683 )
...
* Allow saving / loading checkpoints from cloud paths
Allows saving and loading checkpoints directly from cloud paths like
Amazon S3 (s3://) and Google Cloud Storage (gs://) by using fsspec.
Note: The user will have to install the relevant dependency for each
protocol. Otherwise fsspec will fail and specify which dependency is
missing.
* Append suffix _fsspec to save/load function names
* Add a lower bound to the fsspec dependency
Skips the 0 major version.
* Add missing changes from refactor
* Use fsspec for remaining artifacts
* Add test case with path requiring fsspec
* Avoid writing logs to file unless output_path is local
* Document the possibility of using paths supported by fsspec
* Fix style and lint
* Add missing lint fixes
* Add type annotations to new functions
* Use Coqpit method for converting config to dict
* Fix type annotation in semi-new function
* Add return type for load_fsspec
* Fix bug where fs not always created
* Restore the experiment removal functionality
2021-08-09 18:02:36 +00:00
Eren Gölge
05c75aa9d5
Fix linter issues
2021-07-16 13:37:38 +02:00
Eren Gölge
58cc414477
Fix WaveGrad `test_run`
2021-07-16 13:02:25 +02:00
WeberJulian
25832eb97b
Changes for review
2021-07-15 11:38:45 +02:00
WeberJulian
c79a82ed07
refix linter
2021-07-13 23:12:18 +02:00