mirror of https://github.com/coqui-ai/TTS.git
Update README.md
This commit is contained in:
parent
16561ab40a
commit
c10d47e243
44
README.md
44
README.md
|
@ -1,16 +1,19 @@
|
||||||
# TTS (Work in Progress...)
|
# TTS (Work in Progress...)
|
||||||
TTS targets a Text2Speech engine lightweight in computation with hight quality speech construction.
|
TTS targets a Text2Speech engine lightweight in computation with hight quality speech construction.
|
||||||
|
|
||||||
Here we have pytorch implementation of Tacotron: [A Fully End-to-End Text-To-Speech Synthesis Model](https://arxiv.org/abs/1703.10135) as the start point. We plan to improve the model by the recent updated at the field.
|
Here we have pytorch implementation of Tacotron: [A Fully End-to-End Text-To-Speech Synthesis Model](https://arxiv.org/abs/1703.10135) as the start point. We plan to improve the model by the time with new architectural changes.
|
||||||
|
|
||||||
You can find [here](https://www.evernote.com/shard/s146/sh/9544e7e9-d372-4610-a7b7-3ddcb63d5dac/d01d33837dab625229dec3cfb4cfb887) a brief note pointing possible TTS architectures and their comparisons.
|
You can find [here](http://www.erogol.com/speech-text-deep-learning-architectures/) a brief note pointing possible TTS architectures and their comparisons.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
Highly recommended to use [miniconda](https://conda.io/miniconda.html) for easier installation.
|
Highly recommended to use [miniconda](https://conda.io/miniconda.html) for easier installation.
|
||||||
* python 3.6
|
* python 3.6
|
||||||
* pytorch > 0.2.0
|
* pytorch 0.4
|
||||||
* TODO
|
* librosa
|
||||||
|
* tensorboard
|
||||||
|
* tensorboardX
|
||||||
|
* matplotlib
|
||||||
|
* unidecode
|
||||||
|
|
||||||
## Checkpoints and Audio Samples
|
## Checkpoints and Audio Samples
|
||||||
Checkout [here](https://mycroft.ai/blog/available-voices/#the-human-voice-is-the-most-perfect-instrument-of-all-arvo-part) to compare the samples (except the first) below.
|
Checkout [here](https://mycroft.ai/blog/available-voices/#the-human-voice-is-the-most-perfect-instrument-of-all-arvo-part) to compare the samples (except the first) below.
|
||||||
|
@ -20,7 +23,6 @@ Checkout [here](https://mycroft.ai/blog/available-voices/#the-human-voice-is-the
|
||||||
| [iter-62410](https://drive.google.com/open?id=1pjJNzENL3ZNps9n7k_ktGbpEl6YPIkcZ)| [99d56f7](https://github.com/mozilla/TTS/tree/99d56f7e93ccd7567beb0af8fcbd4d24c48e59e9) | [link](https://soundcloud.com/user-565970875/99d56f7-iter62410 )|
|
| [iter-62410](https://drive.google.com/open?id=1pjJNzENL3ZNps9n7k_ktGbpEl6YPIkcZ)| [99d56f7](https://github.com/mozilla/TTS/tree/99d56f7e93ccd7567beb0af8fcbd4d24c48e59e9) | [link](https://soundcloud.com/user-565970875/99d56f7-iter62410 )|
|
||||||
| Best: [iter-170K](https://drive.google.com/open?id=16L6JbPXj6MSlNUxEStNn28GiSzi4fu1j) | [e00bc66]() |[link](https://soundcloud.com/user-565970875/april-13-2018-07-06pm-e00bc66-iter170k)|
|
| Best: [iter-170K](https://drive.google.com/open?id=16L6JbPXj6MSlNUxEStNn28GiSzi4fu1j) | [e00bc66]() |[link](https://soundcloud.com/user-565970875/april-13-2018-07-06pm-e00bc66-iter170k)|
|
||||||
|
|
||||||
|
|
||||||
## Data
|
## Data
|
||||||
Currently TTS provides data loaders for
|
Currently TTS provides data loaders for
|
||||||
- [LJ Speech](https://keithito.com/LJ-Speech-Dataset/)
|
- [LJ Speech](https://keithito.com/LJ-Speech-Dataset/)
|
||||||
|
@ -58,6 +60,8 @@ Example ```config.json```:
|
||||||
"batch_size": 32,
|
"batch_size": 32,
|
||||||
"eval_batch_size":32,
|
"eval_batch_size":32,
|
||||||
"r": 5,
|
"r": 5,
|
||||||
|
"mk": 0.0, // guidede attention loss weight. if 0 no use
|
||||||
|
"priority_freq": true, // freq range emphasis
|
||||||
|
|
||||||
"griffin_lim_iters": 60,
|
"griffin_lim_iters": 60,
|
||||||
"power": 1.2,
|
"power": 1.2,
|
||||||
|
@ -82,23 +86,17 @@ Best way to test your pretrained network is to use the Notebook under ```noteboo
|
||||||
Any kind of contribution is highly welcome as we are propelled by the open-source spirit. If you like to add or edit things in code, please also consider to write tests to verify your segment so that we can be sure things are on the track as this repo gets bigger.
|
Any kind of contribution is highly welcome as we are propelled by the open-source spirit. If you like to add or edit things in code, please also consider to write tests to verify your segment so that we can be sure things are on the track as this repo gets bigger.
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
- [x] Make the default Tacotron architecture functional with reasonable fidelity.
|
Checkout issues and Project field.
|
||||||
- [ ] Update the architecture with the latest improvements at the field. (e.g. Monotonic Attention, World Vocoder)
|
|
||||||
- Using harmonized teacher forcing proposed by
|
## References
|
||||||
- Update the attention module with a monotonic alternative. (e.g GMM attention, Window based attention)
|
- [Efficient Neural Audio Synthesis](https://arxiv.org/pdf/1802.08435.pdf)
|
||||||
- References:
|
- [Attention-Based models for speech recognition](https://arxiv.org/pdf/1506.07503.pdf)
|
||||||
- [Efficient Neural Audio Synthesis](https://arxiv.org/pdf/1802.08435.pdf)
|
- [Generating Sequences With Recurrent Neural Networks](https://arxiv.org/pdf/1308.0850.pdf)
|
||||||
- [Attention-Based models for speech recognition](https://arxiv.org/pdf/1506.07503.pdf)
|
- [Char2Wav: End-to-End Speech Synthesis](https://openreview.net/pdf?id=B1VWyySKx)
|
||||||
- [Generating Sequences With Recurrent Neural Networks](https://arxiv.org/pdf/1308.0850.pdf)
|
- [VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop](https://arxiv.org/pdf/1707.06588.pdf)
|
||||||
- [Char2Wav: End-to-End Speech Synthesis](https://openreview.net/pdf?id=B1VWyySKx)
|
- [WaveRNN](https://arxiv.org/pdf/1802.08435.pdf)
|
||||||
- [VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop](https://arxiv.org/pdf/1707.06588.pdf)
|
- [Faster WaveNet](https://arxiv.org/abs/1611.09482)
|
||||||
- [ ] Simplify the architecture and push the limits of performance vs efficiency.
|
- [Parallel WaveNet](https://arxiv.org/abs/1711.10433)
|
||||||
- [ ] Improve vocoder part of the network.
|
|
||||||
- Possible Solutions:
|
|
||||||
- WORLD vocoder
|
|
||||||
- [WaveRNN](https://arxiv.org/pdf/1802.08435.pdf)
|
|
||||||
- [Faster WaveNet](https://arxiv.org/abs/1611.09482)
|
|
||||||
- [Parallel WaveNet](https://arxiv.org/abs/1711.10433)
|
|
||||||
|
|
||||||
### Precursor implementations
|
### Precursor implementations
|
||||||
- https://github.com/keithito/tacotron (Dataset and Test processing)
|
- https://github.com/keithito/tacotron (Dataset and Test processing)
|
||||||
|
|
Loading…
Reference in New Issue