--- license: cc-by-nc-sa-4.0 language: - en pipeline_tag: text-to-audio tags: - audiocraft - audiogen - styletts2 - shift-tts - sound - audio-generation - text-to-speech - mimic3 --- SHIFT TTS - StyleTTS2 with Sythetic Speakers (made by another TTS) [](https://shift-europe.eu/) ## # SHIFT TTS / AudioGen Beta version of [SHIFT](https://shift-europe.eu/) TTS tool with [AudioGen soundscapes](https://huggingface.co/dkounadis/artificial-styletts2/discussions/3) - [Analysis of emotion of SHIFT TTS](https://huggingface.co/dkounadis/artificial-styletts2/discussions/2) - [Listen Also foreign languages](https://huggingface.co/dkounadis/artificial-styletts2/discussions/4) synthesized via [MMS TTS](https://huggingface.co/facebook/mms-tts) ## Listen Voices <a href="https://huggingface.co/dkounadis/artificial-styletts2/discussions/1#67854dcbd3e6beb1a78f7f20">Native English</a> / <a href="https://huggingface.co/dkounadis/artificial-styletts2/discussions/1#6783e3b00e7d90facec060c6">Non-native English: Accents</a> / <a href="https://huggingface.co/dkounadis/artificial-styletts2/blob/main/Utils/all_langs.csv">Foreign languages</a> ## [TTS Demo](https://huggingface.co/dkounadis/artificial-styletts2/blob/main/demo.py) ## Flask API <details> <summary> Build virtualenv & run api.py </summary> Above [TTS Demo](https://huggingface.co/dkounadis/artificial-styletts2/blob/main/demo.py) is a standalone script that loads SHIFT TTS & AudioGen model(s) and synthesizes a txt. We also provide a Flask `api.py` that allows faster inference with loading only once the TTS & AudioGen model. Clone ``` git clone https://huggingface.co/dkounadis/artificial-styletts2 ``` Install ``` virtualenv --python=python3 ~/.envs/.my_env source ~/.envs/.my_env/bin/activate cd artificial-styletts2/ pip install -r requirements.txt ``` Flask `tmux-session` ``` CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=0 python api.py ``` Following examples need `api.py` to be running. [Set this IP](https://huggingface.co/dkounadis/artificial-styletts2/blob/main/tts.py#L85) to the IP shown when starting `api.py`. </details> ## Landscape 2 Soundscapes The following needs `api.py` to be already running on a tmux session. ```python # TTS & soundscape - overlay to .mp4 python landscape2soundscape.py ``` For SHIFT demo / Collaboration with [SMB](https://www.smb.museum/home/) - YouTube Videos [](https://youtu.be/SSi3gUO4GtY) [](https://youtu.be/2YjxAPkdXIc) [](https://youtu.be/BhMh02knkco) [](https://youtu.be/a3qk9S87v60) [](https://youtu.be/3M0y9OYzDfU) [](https://youtu.be/OBY666_By1k) [](https://youtu.be/gnGCYLcdLsA) [](https://www.youtube.com/watch?v=Y8QyYUgLaCg) [](https://youtu.be/XDDzxDSrhb0) [](https://youtu.be/I3YYKiUzHpA) # SoundScape Live (iterative) Demo - Paplay Special Flask API for playing sounds live ```python CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=/data/dkounadis/.hf7/ CUDA_VISIBLE_DEVICES=4 python live_api.py ``` Client - Describe any sound with words and it will be played back to you. ```python python live_demo.py # will ask text input & play soundscape ``` # SoundScape (basic) Demo ```python CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=/data/dkounadis/.hf7/ CUDA_VISIBLE_DEVICES=4 python demo.py ``` ## # Audiobook Create audiobook from `.docx`. Listen to it - YouTube [male voice](https://www.youtube.com/watch?v=5-cpf7u18JE) / [female voice](https://www.youtube.com/watch?v=pzrLYCaWD2A) ```python # audiobook will be saved in ./tts_audiobooks python audiobook.py ```