Add model card

README.md

@@ -0,0 +1,172 @@
+---
+language: "en"
+thumbnail:
+tags:
+- ASR
+- CTC
+- Attention
+- pytorch
+license: "apache-2.0"
+datasets:
+- librispeech
+metrics:
+- wer
+- cer
+---
+
+# CRDNN with CTC/Attention and RNNLM trained on LibriSpeech
+
+This repository provides all the necessary tools to perform automatic speech
+recognition from an end-to-end system pretrained on LibriSpeech (EN) within
+SpeechBrain. For a better experience we encourage you to learn more about
+[SpeechBrain](https://speechbrain.github.io). The given ASR model performance are:
+
+| Release | hyperparams file | Test WER | Model link | GPUs |
+|:-------------:|:---------------------------:| -----:| -----:| --------:|
+| 20-05-22 | BPE_1000.yaml | 3.08 | Not Available | 1xV100 32GB |
+| 20-05-22 | BPE_5000.yaml | 2.89 | Not Available | 1xV100 32GB |
+
+## Pipeline description
+
+This ASR system is composed with 3 different but linked blocks:
+1. Tokenizer (unigram) that transforms words into subword units and trained with
+the train transcriptions of LibriSpeech.
+2. Neural language model (RNNLM) trained on the full 10M words dataset.
+3. Acoustic model (CRDNN + CTC/Attention). The CRDNN architecture is made of
+N blocks of convolutional neural networks with normalisation and pooling on the
+frequency domain. Then, a bidirectional LSTM is connected to a final DNN to obtain
+the final acoustic representation that is given to the CTC and attention decoders.
+
+## Intended uses & limitations
+
+This model has been primilarly developed to be run within SpeechBrain as a pretrained ASR model
+for the english language. Thanks to the flexibility of SpeechBrain, any of the 3 blocks
+detailed above can be extracted and connected to you custom pipeline as long as SpeechBrain is
+installed.
+
+## Install SpeechBrain
+
+First of all, please install SpeechBrain with the following command:
+
+```
+pip install \\we hide ! SpeechBrain is still private :p
+```
+
+Please notice that we encourage you to read our tutorials and learn more about
+[SpeechBrain](https://speechbrain.github.io).
+
+### Transcribing your own audio files
+
+```python
+import torch
+import torchaudio
+import speechbrain
+from speechbrain.lobes.pretrained.librispeech.asr_crdnn_ctc_att_rnnlm.acoustic import ASR
+
+asr_model = ASR()
+
+# Make sure your output is sampled at 16 kHz.
+audio_file='path_to_your_audio_file'
+wav, fs = torchaudio.load(audio_file)
+wav_lens = torch.tensor([1]).float()
+
+# Transcribe!
+words, tokens = asr_model.transcribe(wav, wav_lens)
+print(words)
+
+```
+
+### Obtaining encoded features
+
+The SpeechBrain ASR() Class provides an easy way to encode the speech signal
+without running the decoding phase. Hence, one can obtain the output of the
+CRDNN model.
+
+```python
+import torch
+import torchaudio
+import speechbrain
+from speechbrain.lobes.pretrained.librispeech.asr_crdnn_ctc_att_rnnlm.acoustic import ASR
+
+asr_model = ASR()
+
+# Make sure your output is sampled at 16 kHz.
+audio_file='path_to_your_audio_file'
+wav, fs = torchaudio.load(audio_file)
+wav_lens = torch.tensor([1]).float()
+
+# Transcribe!
+words, tokens = asr_model.encode(wav, wav_lens)
+print(words)
+
+```
+
+### Playing with the language model only
+
+Thanks to SpeechBrain lobes, it is feasible to simply instantiate the language
+model to further processing on your custom pipeline:
+
+```python
+import torch
+import speechbrain
+from speechbrain.lobes.pretrained.librispeech.asr_crdnn_ctc_att_rnnlm.lm import LM
+
+lm = LM()
+
+text = "THE CAT IS ON"
+
+# Next word prediction
+encoded_text = lm.tokenizer.encode_as_ids(text)
+encoded_text = torch.Tensor(encoded_text).unsqueeze(0)
+prob_out, _ = lm(encoded_text.to(lm.device))
+index = int(torch.argmax(prob_out[0,-1,:]))
+print(lm.tokenizer.decode(index))
+
+# Text generation
+encoded_text = torch.tensor([0, 2]) # bos token + the
+encoded_text = encoded_text.unsqueeze(0).to(lm.device)
+for i in range(19):
+  prob_out, _ = lm(encoded_text)
+  index = torch.argmax(prob_out[0,-1,:]).unsqueeze(0)
+  encoded_text = torch.cat([encoded_text, index.unsqueeze(0)], dim=1)
+encoded_text = encoded_text[0,1:].tolist()
+print(lm.tokenizer.decode(encoded_text))
+
+```
+
+### Playing with the tokenizer only
+
+In the same manner as for the language model, one can isntantiate the tokenizer
+only with the corresponding lobes in SpeechBrain.  
+
+```python
+import speechbrain
+from speechbrain.lobes.pretrained.librispeech.asr_crdnn_ctc_att_rnnlm.tokenizer import tokenizer
+
+# HuggingFace paths to download the pretrained models
+token_file = 'tokenizer/1000_unigram.model'
+model_name = 'sb/asr-crdnn-librispeech'
+save_dir = 'model_checkpoints'
+
+text = "THE CAT IS ON THE TABLE"
+
+tokenizer = tokenizer(token_file, model_name, save_dir)
+
+# Tokenize!
+print(tokenizer.spm.encode(text))
+print(tokenizer.spm.encode(text, out_type='str'))
+
+```
+
+#### Referencing SpeechBrain
+
+```
+@misc{SB2021,
+    author = {Ravanelli, Mirco and Parcollet, Titouan and Rouhe, Aku and Plantinga, Peter and Rastorgueva, Elena and Lugosch, Loren and Dawalatabad, Nauman and Ju-Chieh, Chou and Heba, Abdel and Grondin, Francois and Aris, William and Liao, Chien-Feng and Cornell, Samuele and Yeh, Sung-Lin and Na, Hwidong and Gao, Yan and Fu, Szu-Wei and Subakan, Cem and De Mori, Renato and Bengio, Yoshua },
+    title = {SpeechBrain},
+    year = {2021},
+    publisher = {GitHub},
+    journal = {GitHub repository},
+    howpublished = {\url{https://github.com/speechbrain/speechbrain}},
+  }
+```