metadata

license: cc-by-nc-sa-4.0
language:
  - en
pipeline_tag: text-to-audio
tags:
  - audiocraft
  - audiogen
  - styletts2
  - shift-tts
  - sound
  - audio-generation
  - text-to-speech
  - mimic3

SHIFT TTS - StyleTTS2 with Sythetic Speakers (made by another TTS)

SHIFT TTS / AudioGen

Beta version of SHIFT TTS tool with AudioGen soundscapes

Analysis of emotion of SHIFT TTS
Listen Also foreign languages synthesized via MMS TTS

Listen Voices

Native English / Non-native English: Accents / Foreign languages

TTS Demo

Flask API

Build virtualenv & run api.py

Above TTS Demo is a standalone script that loads SHIFT TTS & AudioGen model(s) and synthesizes a txt. We also provide a Flask api.py that allows faster inference with loading only once the TTS & AudioGen model.

Clone

git clone https://huggingface.co/dkounadis/artificial-styletts2

Install

virtualenv --python=python3 ~/.envs/.my_env
source ~/.envs/.my_env/bin/activate
cd artificial-styletts2/
pip install -r requirements.txt

Flask tmux-session

CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=0 python api.py

Following examples need api.py to be running. Set this IP to the IP shown when starting api.py.

Landscape 2 Soundscapes

The following needs api.py to be already running on a tmux session.

# TTS & soundscape - overlay to .mp4
python landscape2soundscape.py

For SHIFT demo / Collaboration with SMB

YouTube Videos

SoundScape Live (iterative) Demo - Paplay

Special Flask API for playing sounds live

CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=/data/dkounadis/.hf7/ CUDA_VISIBLE_DEVICES=4 python live_api.py

Client - Describe any sound with words and it will be played back to you.

python live_demo.py  # will ask text input & play soundscape

SoundScape (basic) Demo

CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=/data/dkounadis/.hf7/ CUDA_VISIBLE_DEVICES=4 python demo.py

Audiobook

Create audiobook from .docx. Listen to it - YouTube male voice / female voice

#  audiobook will be saved in ./tts_audiobooks
python audiobook.py