Hugging Face's logo

qualcomm
/
Whisper-Small-En

Automatic Speech Recognition
PyTorch
TF Lite
ONNX
foundation
android
Model card Files Community
qaihm-bot commited on
Commit
405fc10
·
verified ·
1 Parent(s): fe083e9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +67 -28
README.md CHANGED
@@ -15,7 +15,7 @@ tags:
15
 
16
  OpenAI’s Whisper ASR (Automatic Speech Recognition) model is a state-of-the-art system designed for transcribing spoken language into written text. It exhibits robust performance in realistic, noisy environments, making it highly reliable for real-world applications. Specifically, it excels in long-form transcription, capable of accurately transcribing audio clips up to 30 seconds long. Time to the first token is the encoder's latency, while time to each additional token is decoder's latency, where we assume a mean decoded length specified below.
17
 
18
- This model is an implementation of Whisper-Small-En found [here](https://github.com/openai/whisper/tree/main).
19
  This repository provides scripts to run Whisper-Small-En on Qualcomm® devices.
20
  More details on model performance across various devices, can be found
21
  [here](https://aihub.qualcomm.com/models/whisper_small_en).
@@ -33,17 +33,50 @@ More details on model performance across various devices, can be found
33
  - Number of parameters (WhisperDecoder): 139M
34
  - Model size (WhisperDecoder): 531 MB
35
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
 
37
 
38
 
39
- | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
40
- | ---|---|---|---|---|---|---|---|
41
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 707.067 ms | 18 - 418 MB | FP16 | GPU | [WhisperEncoder.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite)
42
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | TFLite | 25.831 ms | 16 - 20 MB | FP16 | NPU | [WhisperDecoder.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite)
43
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 1196.02 ms | 0 - 203 MB | FP16 | NPU | [WhisperEncoder.so](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.so)
44
- | Samsung Galaxy S23 Ultra (Android 13) | Snapdragon® 8 Gen 2 | QNN Model Library | 12.11 ms | 55 - 120 MB | FP16 | NPU | [WhisperDecoder.so](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.so)
45
-
46
-
47
 
48
  ## Installation
49
 
@@ -99,23 +132,25 @@ device. This script does the following:
99
  ```bash
100
  python -m qai_hub_models.models.whisper_small_en.export
101
  ```
102
-
103
  ```
104
- Profile Job summary of WhisperEncoder
105
- --------------------------------------------------
106
- Device: Snapdragon X Elite CRD (11)
107
- Estimated Inference Time: 641.24 ms
108
- Estimated Peak Memory Range: 0.46-0.46 MB
109
- Compute Units: NPU (1329) | Total (1329)
110
-
111
- Profile Job summary of WhisperDecoder
112
- --------------------------------------------------
113
- Device: Snapdragon X Elite CRD (11)
114
- Estimated Inference Time: 10.42 ms
115
- Estimated Peak Memory Range: 60.75-60.75 MB
116
- Compute Units: NPU (2255) | Total (2255)
117
-
118
-
 
 
 
119
  ```
120
 
121
 
@@ -243,15 +278,19 @@ provides instructions on how to use the `.so` shared library in an Android appl
243
  Get more details on Whisper-Small-En's performance across various devices [here](https://aihub.qualcomm.com/models/whisper_small_en).
244
  Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
245
 
 
246
  ## License
247
- - The license for the original implementation of Whisper-Small-En can be found
248
- [here](https://github.com/openai/whisper/blob/main/LICENSE).
249
- - The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
 
250
 
251
  ## References
252
  * [Robust Speech Recognition via Large-Scale Weak Supervision](https://cdn.openai.com/papers/whisper.pdf)
253
  * [Source Model Implementation](https://github.com/openai/whisper/tree/main)
254
 
 
 
255
  ## Community
256
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
257
  * For questions or feedback please [reach out to us](mailto:[email protected]).
 
15
 
16
  OpenAI’s Whisper ASR (Automatic Speech Recognition) model is a state-of-the-art system designed for transcribing spoken language into written text. It exhibits robust performance in realistic, noisy environments, making it highly reliable for real-world applications. Specifically, it excels in long-form transcription, capable of accurately transcribing audio clips up to 30 seconds long. Time to the first token is the encoder's latency, while time to each additional token is decoder's latency, where we assume a mean decoded length specified below.
17
 
18
+ This model is an implementation of Whisper-Small-En found [here]({source_repo}).
19
  This repository provides scripts to run Whisper-Small-En on Qualcomm® devices.
20
  More details on model performance across various devices, can be found
21
  [here](https://aihub.qualcomm.com/models/whisper_small_en).
 
33
  - Number of parameters (WhisperDecoder): 139M
34
  - Model size (WhisperDecoder): 531 MB
35
 
36
+ | Model | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
37
+ |---|---|---|---|---|---|---|---|---|
38
+ | WhisperEncoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 696.399 ms | 81 - 473 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
39
+ | WhisperEncoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 854.116 ms | 0 - 207 MB | FP16 | NPU | [Whisper-Small-En.so](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.so) |
40
+ | WhisperEncoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 549.997 ms | 111 - 197 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
41
+ | WhisperEncoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 693.407 ms | 0 - 839 MB | FP16 | NPU | [Whisper-Small-En.so](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.so) |
42
+ | WhisperEncoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 861.217 ms | 115 - 4307 MB | FP16 | NPU | [Whisper-Small-En.onnx](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.onnx) |
43
+ | WhisperEncoder | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 689.836 ms | 40 - 422 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
44
+ | WhisperEncoder | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 694.52 ms | 1 - 2 MB | FP16 | NPU | Use Export Script |
45
+ | WhisperEncoder | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 683.626 ms | 83 - 467 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
46
+ | WhisperEncoder | SA8255 (Proxy) | SA8255P Proxy | QNN | 712.687 ms | 1 - 31 MB | FP16 | NPU | Use Export Script |
47
+ | WhisperEncoder | SA8775 (Proxy) | SA8775P Proxy | TFLITE | 673.361 ms | 9 - 223 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
48
+ | WhisperEncoder | SA8775 (Proxy) | SA8775P Proxy | QNN | 695.416 ms | 1 - 3 MB | FP16 | NPU | Use Export Script |
49
+ | WhisperEncoder | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 674.483 ms | 110 - 475 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
50
+ | WhisperEncoder | SA8650 (Proxy) | SA8650P Proxy | QNN | 691.155 ms | 6 - 8 MB | FP16 | NPU | Use Export Script |
51
+ | WhisperEncoder | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 986.018 ms | 110 - 207 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
52
+ | WhisperEncoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 536.818 ms | 109 - 137 MB | FP16 | GPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.tflite) |
53
+ | WhisperEncoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 551.075 ms | 0 - 910 MB | FP16 | NPU | Use Export Script |
54
+ | WhisperEncoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 635.179 ms | 117 - 2773 MB | FP16 | NPU | [Whisper-Small-En.onnx](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.onnx) |
55
+ | WhisperEncoder | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 526.589 ms | 0 - 0 MB | FP16 | NPU | Use Export Script |
56
+ | WhisperEncoder | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 1357.587 ms | 449 - 449 MB | FP16 | NPU | [Whisper-Small-En.onnx](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperEncoder.onnx) |
57
+ | WhisperDecoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 26.563 ms | 16 - 19 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
58
+ | WhisperDecoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 11.991 ms | 61 - 127 MB | FP16 | NPU | [Whisper-Small-En.so](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.so) |
59
+ | WhisperDecoder | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 56.19 ms | 121 - 124 MB | FP16 | NPU | [Whisper-Small-En.onnx](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.onnx) |
60
+ | WhisperDecoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 21.257 ms | 16 - 1128 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
61
+ | WhisperDecoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 9.77 ms | 39 - 134 MB | FP16 | NPU | [Whisper-Small-En.so](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.so) |
62
+ | WhisperDecoder | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 47.118 ms | 114 - 1592 MB | FP16 | NPU | [Whisper-Small-En.onnx](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.onnx) |
63
+ | WhisperDecoder | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 24.89 ms | 16 - 19 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
64
+ | WhisperDecoder | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 12.378 ms | 61 - 62 MB | FP16 | NPU | Use Export Script |
65
+ | WhisperDecoder | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 25.507 ms | 14 - 17 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
66
+ | WhisperDecoder | SA8255 (Proxy) | SA8255P Proxy | QNN | 12.827 ms | 61 - 66 MB | FP16 | NPU | Use Export Script |
67
+ | WhisperDecoder | SA8775 (Proxy) | SA8775P Proxy | TFLITE | 24.833 ms | 15 - 18 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
68
+ | WhisperDecoder | SA8775 (Proxy) | SA8775P Proxy | QNN | 12.718 ms | 61 - 66 MB | FP16 | NPU | Use Export Script |
69
+ | WhisperDecoder | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 25.027 ms | 16 - 19 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
70
+ | WhisperDecoder | SA8650 (Proxy) | SA8650P Proxy | QNN | 12.546 ms | 61 - 62 MB | FP16 | NPU | Use Export Script |
71
+ | WhisperDecoder | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 27.823 ms | 16 - 1104 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
72
+ | WhisperDecoder | QCS8450 (Proxy) | QCS8450 Proxy | QNN | 14.222 ms | 57 - 160 MB | FP16 | NPU | Use Export Script |
73
+ | WhisperDecoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 15.389 ms | 15 - 262 MB | FP16 | NPU | [Whisper-Small-En.tflite](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.tflite) |
74
+ | WhisperDecoder | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 7.56 ms | 61 - 195 MB | FP16 | NPU | Use Export Script |
75
+ | WhisperDecoder | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 10.849 ms | 61 - 61 MB | FP16 | NPU | Use Export Script |
76
+ | WhisperDecoder | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 49.274 ms | 232 - 232 MB | FP16 | NPU | [Whisper-Small-En.onnx](https://huggingface.co/qualcomm/Whisper-Small-En/blob/main/WhisperDecoder.onnx) |
77
 
78
 
79
 
 
 
 
 
 
 
 
 
80
 
81
  ## Installation
82
 
 
132
  ```bash
133
  python -m qai_hub_models.models.whisper_small_en.export
134
  ```
 
135
  ```
136
+ Profiling Results
137
+ ------------------------------------------------------------
138
+ WhisperEncoder
139
+ Device : Samsung Galaxy S23 (13)
140
+ Runtime : TFLITE
141
+ Estimated inference time (ms) : 696.4
142
+ Estimated peak memory usage (MB): [81, 473]
143
+ Total # Ops : 911
144
+ Compute Unit(s) : GPU (900 ops) CPU (11 ops)
145
+
146
+ ------------------------------------------------------------
147
+ WhisperDecoder
148
+ Device : Samsung Galaxy S23 (13)
149
+ Runtime : TFLITE
150
+ Estimated inference time (ms) : 26.6
151
+ Estimated peak memory usage (MB): [16, 19]
152
+ Total # Ops : 2573
153
+ Compute Unit(s) : NPU (2573 ops)
154
  ```
155
 
156
 
 
278
  Get more details on Whisper-Small-En's performance across various devices [here](https://aihub.qualcomm.com/models/whisper_small_en).
279
  Explore all available models on [Qualcomm® AI Hub](https://aihub.qualcomm.com/)
280
 
281
+
282
  ## License
283
+ * The license for the original implementation of Whisper-Small-En can be found [here](https://github.com/openai/whisper/blob/main/LICENSE).
284
+ * The license for the compiled assets for on-device deployment can be found [here](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/Qualcomm+AI+Hub+Proprietary+License.pdf)
285
+
286
+
287
 
288
  ## References
289
  * [Robust Speech Recognition via Large-Scale Weak Supervision](https://cdn.openai.com/papers/whisper.pdf)
290
  * [Source Model Implementation](https://github.com/openai/whisper/tree/main)
291
 
292
+
293
+
294
  ## Community
295
  * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
296
  * For questions or feedback please [reach out to us](mailto:[email protected]).