{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "5cefac89" }, "source": [ "# Fine-tune Whisper (large) with LoRA & BNB powerd by PEFT β‘οΈ" ], "id": "5cefac89" }, { "cell_type": "markdown", "metadata": { "id": "090fa3ed" }, "source": [ "A one size fits all notebook, to fine-tune Whisper (large) on a consumer GPU with less than 8GB GPU VRAM, all with comparable performance to full-finetuning. β‘οΈ\n", "\n", "We present a step-by-step guide on how to fine-tune Whisper with Common Voice 13.0 dataset using π€ Transformers and PEFT. In this Colab, we leverage `PEFT` and `bitsandbytes` to train a `whisper-large-v2` checkpoint seamlessly with a free T4 GPU (16 GB VRAM).\n", "\n", "For more details on Whisper fine-tuning, datasets and metrics, refer to Sanchit Gandhi's brilliant blogpost: [Fine-Tune Whisper For Multilingual ASR with π€ Transformers](https://huggingface.co/blog/fine-tune-whisper)\n", "\n", "## Why Parameter Efficient Fine Tuning ([PEFT](https://github.com/huggingface/peft))?\n", "\n", "As the model size continue to increase, fine tuning a model has become both computationally expensive and storage heavy. For example, a `Whisper-large-v2` model requires ~24GB of GPU VRAM to fine-tune for full fine-tuning and requires ~7 GB of storage for each fine-tuned storage. For low-resource environments this becomes quite a bottleneck and often near impossible to get meaningful results.\n", "\n", "Cue, PEFT, with PEFT you can tackle this bottleneck head on. PEFT approaches (like Low Rank Adaptation) only fine-tune a small number of (extra) model parameters while freezing most parameters of the pretrained model, thereby greatly decreasing the computational and storage costs. We've observed that it also overcomes the issues of catastrophic forgetting, a behaviour observed during the full finetuning of large models.\n", "\n", "### Aha! So wait, what's this LoRA thing?\n", "\n", "PEFT comes out-of-the-box with multiple parameter efficient techniques. One such technique is [Low Rank Adaptation or LoRA](https://github.com/microsoft/LoRA). LoRA freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture. This greatly reduces the number of trainable parameters for downstream tasks.\n", "\n", "LoRA performs on-par or better than fine-tuning in model quality despite having fewer trainable parameters, a higher training throughput, and, unlike adapters, no additional inference latency.\n", "\n", "### That's all cool, but show me the numbers?\n", "\n", "Don't worry, we got ya! We ran multiple experiments to compare a full fine-tuning of Whisper-large-v2 checkpoint and that with PEFT, here's what we found:\n", "\n", "1. We were able to fine-tune a 1.6B parameter model with less than 8GB GPU VRAM. π€―\n", "2. With significantly less number of traininable parameters, we were able to fit almost **5x** more batch size. π\n", "3. The resultant checkpoint were less than 1% the size of the original model, ~60MB (i.e. 1% the size of orignal model) π\n", "\n", "To make things even better, all of this comes with minimal changes to the existing π€ transformers Whisper inference codebase.\n", "\n", "### Curious to test this out for yourself? Follow along!" ], "id": "090fa3ed" }, { "cell_type": "markdown", "metadata": { "id": "625e47a0" }, "source": [ "## Prepare Environment\n", "\n", "We'll employ several popular Python packages to fine-tune the Whisper model.\n", "We'll use `datasets` to download and prepare our training data and\n", "`transformers` to load and train our Whisper model. We'll also require\n", "the `librosa` package to pre-process audio files, `evaluate` and `jiwer` to\n", "assess the performance of our model. Finally, we'll\n", "use `PEFT`, `bitsandbytes`, `accelerate` to prepare and fine-tune the model with LoRA." ], "id": "625e47a0" }, { "cell_type": "code", "source": [ "!add-apt-repository -y ppa:jonathonf/ffmpeg-4\n", "!apt update\n", "!apt install -y ffmpeg" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "RLNCEM9IPH-q", "outputId": "4ac9bd68-52d2-492b-963a-8ea7d504a937", "collapsed": true }, "id": "RLNCEM9IPH-q", "execution_count": 1, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Repository: 'deb https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu/ jammy main'\n", "Description:\n", "Backport of FFmpeg 4 and associated libraries. Now includes AOM/AV1 support!\n", "\n", "FDK AAC is not compatible with GPL and FFmpeg can't be redistributed with it included. Please don't ask for it to be added to this public PPA.\n", "\n", "---\n", "\n", "PPA supporters:\n", "\n", "BigBlueButton (https://bigbluebutton.org)\n", "\n", "---\n", "\n", "Donate to FFMPEG: https://ffmpeg.org/donations.html\n", "Donate to Debian: https://www.debian.org/donations\n", "Donate to this PPA: https://ko-fi.com/jonathonf\n", "More info: https://launchpad.net/~jonathonf/+archive/ubuntu/ffmpeg-4\n", "Adding repository.\n", "Adding deb entry to /etc/apt/sources.list.d/jonathonf-ubuntu-ffmpeg-4-jammy.list\n", "Adding disabled deb-src entry to /etc/apt/sources.list.d/jonathonf-ubuntu-ffmpeg-4-jammy.list\n", "Adding key to /etc/apt/trusted.gpg.d/jonathonf-ubuntu-ffmpeg-4.gpg with fingerprint 4AB0F789CBA31744CC7DA76A8CF63AD3F06FC659\n", "Get:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease [3,626 B]\n", "Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease\n", "Hit:3 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease\n", "Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB]\n", "Hit:5 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease\n", "Hit:6 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease\n", "Ign:7 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy InRelease\n", "Hit:8 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease\n", "Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,453 kB]\n", "Err:10 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release\n", " 404 Not Found [IP: 185.125.190.80 443]\n", "Hit:11 http://archive.ubuntu.com/ubuntu jammy InRelease\n", "Get:12 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB]\n", "Get:13 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,900 kB]\n", "Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1,090 kB]\n", "Hit:15 http://archive.ubuntu.com/ubuntu jammy-backports InRelease\n", "Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,173 kB]\n", "Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,525 kB]\n", "Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,391 kB]\n", "Reading package lists... Done\n", "E: The repository 'https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release' does not have a Release file.\n", "N: Updating from such a repository can't be done securely, and is therefore disabled by default.\n", "N: See apt-secure(8) manpage for repository creation and user configuration details.\n", "Hit:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease\n", "Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease\n", "Hit:3 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease\n", "Hit:4 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease\n", "Hit:5 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease\n", "Ign:6 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy InRelease\n", "Hit:7 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease\n", "Err:8 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release\n", " 404 Not Found [IP: 185.125.190.80 443]\n", "Hit:9 http://security.ubuntu.com/ubuntu jammy-security InRelease\n", "Hit:10 http://archive.ubuntu.com/ubuntu jammy InRelease\n", "Hit:11 http://archive.ubuntu.com/ubuntu jammy-updates InRelease\n", "Hit:12 http://archive.ubuntu.com/ubuntu jammy-backports InRelease\n", "Reading package lists... Done\n", "\u001b[1;31mE: \u001b[0mThe repository 'https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release' does not have a Release file.\u001b[0m\n", "\u001b[33mN: \u001b[0mUpdating from such a repository can't be done securely, and is therefore disabled by default.\u001b[0m\n", "\u001b[33mN: \u001b[0mSee apt-secure(8) manpage for repository creation and user configuration details.\u001b[0m\n", "Reading package lists... Done\n", "Building dependency tree... Done\n", "Reading state information... Done\n", "ffmpeg is already the newest version (7:4.4.2-0ubuntu0.22.04.1).\n", "0 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.\n" ] } ] }, { "cell_type": "code", "source": [ "!pip install datasets>=2.6.1\n", "!pip install git+https://github.com/huggingface/transformers\n", "!pip install librosa\n", "!pip install evaluate>=0.30\n", "!pip install jiwer\n", "!pip install -q bitsandbytes datasets accelerate loralib\n", "!pip install -q git+https://github.com/huggingface/transformers.git@main git+https://github.com/huggingface/peft.git@main" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "z-rH2K1IL91f", "outputId": "441e409d-34de-4911-a601-b36cf3aac657", "collapsed": true }, "id": "z-rH2K1IL91f", "execution_count": 2, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", "google-colab 1.0.0 requires requests==2.31.0, but you have requests 2.32.3 which is incompatible.\u001b[0m\u001b[31m\n", "\u001b[0mCollecting git+https://github.com/huggingface/transformers\n", " Cloning https://github.com/huggingface/transformers to /tmp/pip-req-build-semezgcr\n", " Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers /tmp/pip-req-build-semezgcr\n", " Resolved https://github.com/huggingface/transformers to commit 348e2294ac1a3cc744a93f04ab1f886439ee90d3\n", " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (3.14.0)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.23.2 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (0.23.3)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (1.25.2)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (24.1)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (6.0.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (2024.5.15)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (2.32.3)\n", "Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (0.19.1)\n", "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (0.4.3)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.42.0.dev0) (4.66.4)\n", "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.23.2->transformers==4.42.0.dev0) (2023.6.0)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.23.2->transformers==4.42.0.dev0) (4.12.2)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.42.0.dev0) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.42.0.dev0) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.42.0.dev0) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.42.0.dev0) (2024.6.2)\n", "Building wheels for collected packages: transformers\n", " Building wheel for transformers (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for transformers: filename=transformers-4.42.0.dev0-py3-none-any.whl size=9157040 sha256=ebee4823e7169c91a3db04225c66dd2050cf35c732eeaf0c400fd71fcf63f624\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-8l57bp2z/wheels/c0/14/d6/6c9a5582d2ac191ec0a483be151a4495fe1eb2a6706ca49f1b\n", "Successfully built transformers\n", "Installing collected packages: transformers\n", " Attempting uninstall: transformers\n", " Found existing installation: transformers 4.41.2\n", " Uninstalling transformers-4.41.2:\n", " Successfully uninstalled transformers-4.41.2\n", "Successfully installed transformers-4.42.0.dev0\n", "Requirement already satisfied: librosa in /usr/local/lib/python3.10/dist-packages (0.10.2.post1)\n", "Requirement already satisfied: audioread>=2.1.9 in /usr/local/lib/python3.10/dist-packages (from librosa) (3.0.1)\n", "Requirement already satisfied: numpy!=1.22.0,!=1.22.1,!=1.22.2,>=1.20.3 in /usr/local/lib/python3.10/dist-packages (from librosa) (1.25.2)\n", "Requirement already satisfied: scipy>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from librosa) (1.11.4)\n", "Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.10/dist-packages (from librosa) (1.2.2)\n", "Requirement already satisfied: joblib>=0.14 in /usr/local/lib/python3.10/dist-packages (from librosa) (1.4.2)\n", "Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.10/dist-packages (from librosa) (4.4.2)\n", "Requirement already satisfied: numba>=0.51.0 in /usr/local/lib/python3.10/dist-packages (from librosa) (0.58.1)\n", "Requirement already satisfied: soundfile>=0.12.1 in /usr/local/lib/python3.10/dist-packages (from librosa) (0.12.1)\n", "Requirement already satisfied: pooch>=1.1 in /usr/local/lib/python3.10/dist-packages (from librosa) (1.8.2)\n", "Requirement already satisfied: soxr>=0.3.2 in /usr/local/lib/python3.10/dist-packages (from librosa) (0.3.7)\n", "Requirement already satisfied: typing-extensions>=4.1.1 in /usr/local/lib/python3.10/dist-packages (from librosa) (4.12.2)\n", "Requirement already satisfied: lazy-loader>=0.1 in /usr/local/lib/python3.10/dist-packages (from librosa) (0.4)\n", "Requirement already satisfied: msgpack>=1.0 in /usr/local/lib/python3.10/dist-packages (from librosa) (1.0.8)\n", "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from lazy-loader>=0.1->librosa) (24.1)\n", "Requirement already satisfied: llvmlite<0.42,>=0.41.0dev0 in /usr/local/lib/python3.10/dist-packages (from numba>=0.51.0->librosa) (0.41.1)\n", "Requirement already satisfied: platformdirs>=2.5.0 in /usr/local/lib/python3.10/dist-packages (from pooch>=1.1->librosa) (4.2.2)\n", "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from pooch>=1.1->librosa) (2.32.3)\n", "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=0.20.0->librosa) (3.5.0)\n", "Requirement already satisfied: cffi>=1.0 in /usr/local/lib/python3.10/dist-packages (from soundfile>=0.12.1->librosa) (1.16.0)\n", "Requirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.0->soundfile>=0.12.1->librosa) (2.22)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->pooch>=1.1->librosa) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->pooch>=1.1->librosa) (3.7)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->pooch>=1.1->librosa) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->pooch>=1.1->librosa) (2024.6.2)\n", "Collecting jiwer\n", " Downloading jiwer-3.0.4-py3-none-any.whl (21 kB)\n", "Requirement already satisfied: click<9.0.0,>=8.1.3 in /usr/local/lib/python3.10/dist-packages (from jiwer) (8.1.7)\n", "Collecting rapidfuzz<4,>=3 (from jiwer)\n", " Downloading rapidfuzz-3.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)\n", "\u001b[2K \u001b[90mββββββββββββββββββββββββββββββββββββββββ\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m44.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hInstalling collected packages: rapidfuzz, jiwer\n", "Successfully installed jiwer-3.0.4 rapidfuzz-3.9.3\n", "\u001b[2K \u001b[90mββββββββββββββββββββββββββββββββββββββββ\u001b[0m \u001b[32m119.8/119.8 MB\u001b[0m \u001b[31m13.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90mββββββββββββββββββββββββββββββββββββββββ\u001b[0m \u001b[32m309.4/309.4 kB\u001b[0m \u001b[31m36.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90mββββββββββββββββββββββββββββββββββββββββ\u001b[0m \u001b[32m21.3/21.3 MB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", " Building wheel for peft (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n" ] } ] }, { "cell_type": "code", "source": [ "!add-apt-repository -y ppa:jonathonf/ffmpeg-4\n", "!apt update\n", "!apt install -y ffmpeg" ], "metadata": { "id": "66iVrvlALjZS", "colab": { "base_uri": "https://localhost:8080/" }, "collapsed": true, "outputId": "a8a35837-cf12-4cfd-d3cb-aa6bad0d50d5" }, "id": "66iVrvlALjZS", "execution_count": 3, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Repository: 'deb https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu/ jammy main'\n", "Description:\n", "Backport of FFmpeg 4 and associated libraries. Now includes AOM/AV1 support!\n", "\n", "FDK AAC is not compatible with GPL and FFmpeg can't be redistributed with it included. Please don't ask for it to be added to this public PPA.\n", "\n", "---\n", "\n", "PPA supporters:\n", "\n", "BigBlueButton (https://bigbluebutton.org)\n", "\n", "---\n", "\n", "Donate to FFMPEG: https://ffmpeg.org/donations.html\n", "Donate to Debian: https://www.debian.org/donations\n", "Donate to this PPA: https://ko-fi.com/jonathonf\n", "More info: https://launchpad.net/~jonathonf/+archive/ubuntu/ffmpeg-4\n", "Adding repository.\n", "Found existing deb entry in /etc/apt/sources.list.d/jonathonf-ubuntu-ffmpeg-4-jammy.list\n", "Adding deb entry to /etc/apt/sources.list.d/jonathonf-ubuntu-ffmpeg-4-jammy.list\n", "Found existing deb-src entry in /etc/apt/sources.list.d/jonathonf-ubuntu-ffmpeg-4-jammy.list\n", "Adding disabled deb-src entry to /etc/apt/sources.list.d/jonathonf-ubuntu-ffmpeg-4-jammy.list\n", "Adding key to /etc/apt/trusted.gpg.d/jonathonf-ubuntu-ffmpeg-4.gpg with fingerprint 4AB0F789CBA31744CC7DA76A8CF63AD3F06FC659\n", "Hit:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease\n", "Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease\n", "Hit:3 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease\n", "Hit:4 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease\n", "Hit:5 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease\n", "Ign:6 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy InRelease\n", "Hit:7 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease\n", "Err:8 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release\n", " 404 Not Found [IP: 185.125.190.80 443]\n", "Ign:9 http://security.ubuntu.com/ubuntu jammy-security InRelease\n", "Ign:9 http://security.ubuntu.com/ubuntu jammy-security InRelease\n", "Ign:9 http://security.ubuntu.com/ubuntu jammy-security InRelease\n", "Err:9 http://security.ubuntu.com/ubuntu jammy-security InRelease\n", " Could not connect to security.ubuntu.com:80 (91.189.91.82), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.83), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.81), connection timed out\n", "Hit:10 http://archive.ubuntu.com/ubuntu jammy InRelease\n", "Hit:11 http://archive.ubuntu.com/ubuntu jammy-updates InRelease\n", "Hit:12 http://archive.ubuntu.com/ubuntu jammy-backports InRelease\n", "Reading package lists... Done\n", "E: The repository 'https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release' does not have a Release file.\n", "N: Updating from such a repository can't be done securely, and is therefore disabled by default.\n", "N: See apt-secure(8) manpage for repository creation and user configuration details.\n", "Hit:1 https://cloud.r-project.org/bin/linux/ubuntu jammy-cran40/ InRelease\n", "Hit:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease\n", "Hit:3 https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu jammy InRelease\n", "Hit:4 https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu jammy InRelease\n", "Hit:5 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu jammy InRelease\n", "Ign:6 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy InRelease\n", "Hit:7 https://ppa.launchpadcontent.net/ubuntugis/ppa/ubuntu jammy InRelease\n", "Err:8 https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release\n", " 404 Not Found [IP: 185.125.190.80 443]\n", "Hit:9 http://archive.ubuntu.com/ubuntu jammy InRelease\n", "Hit:10 http://archive.ubuntu.com/ubuntu jammy-updates InRelease\n", "Hit:11 http://archive.ubuntu.com/ubuntu jammy-backports InRelease\n", "Hit:12 http://security.ubuntu.com/ubuntu jammy-security InRelease\n", "Reading package lists... Done\n", "\u001b[1;31mE: \u001b[0mThe repository 'https://ppa.launchpadcontent.net/jonathonf/ffmpeg-4/ubuntu jammy Release' does not have a Release file.\u001b[0m\n", "\u001b[33mN: \u001b[0mUpdating from such a repository can't be done securely, and is therefore disabled by default.\u001b[0m\n", "\u001b[33mN: \u001b[0mSee apt-secure(8) manpage for repository creation and user configuration details.\u001b[0m\n", "Reading package lists... Done\n", "Building dependency tree... Done\n", "Reading state information... Done\n", "ffmpeg is already the newest version (7:4.4.2-0ubuntu0.22.04.1).\n", "0 upgraded, 0 newly installed, 0 to remove and 45 not upgraded.\n" ] } ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "r_Ivl7qlX0dz" }, "outputs": [], "source": [], "id": "r_Ivl7qlX0dz" }, { "cell_type": "markdown", "metadata": { "id": "2MvOaXjUjK71" }, "source": [ "With the environment now set up, let's try to secure a decent GPU for our Colab! Unfortunately, it's becoming much harder to get access to a good GPU with the free version of Google Colab. However, with Google Colab Pro one should have no issues in being allocated a V100 or P100 GPU.\n", "\n", "To get a GPU, click _Runtime_ -> _Change runtime type_, then change _Hardware accelerator_ from _None_ to _GPU_.\n", "\n", "We can verify that we've been assigned a GPU and view its specifications:" ], "id": "2MvOaXjUjK71" }, { "cell_type": "markdown", "metadata": { "id": "6WwnavbBuezQ" }, "source": [ "Alrighty! Let's configure our environment to ensure it uses the GPU provided by Colab to us." ], "id": "6WwnavbBuezQ" }, { "cell_type": "markdown", "metadata": { "id": "8a528c1a" }, "source": [ "We strongly advise you to upload model checkpoints directly the [Hugging Face Hub](https://huggingface.co/)\n", "whilst training. The Hub provides:\n", "- Integrated version control: you can be sure that no model checkpoint is lost during training.\n", "- Tensorboard logs: track important metrics over the course of training.\n", "- Model cards: document what a model does and its intended use cases.\n", "- Community: an easy way to share and collaborate with the community!\n", "\n", "Linking the notebook to the Hub is straightforward - it simply requires entering your Hub authentication token when prompted. Find your Hub authentication token [here](https://huggingface.co/settings/tokens):" ], "id": "8a528c1a" }, { "cell_type": "code", "execution_count": 5, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 160, "referenced_widgets": [ "d845d2eb6aff4f0e9852790153eec4d4", "3726bff6a04b4b09b9e2f9d98af6a49b", "990b768dee5340e1a45ec8400ccee308", "e86df5cd5a47411293a48310f5d076dd", "18ae9937340748f08ee9755a8d2c87d4", "15de19a49cfa4b2f85ac710341475c47", "9c7d61f97b5a4073ad448a8ebc9739eb", "78361d6ab74248538ab7f165dbdf3857", "f2df72d295a644fabeb15adbc4516e51", "18b974e332ca480e872cdb83777dcf82", "a6759f8e199a42aebb6ad93c855a7a12", "cc9587ee4da7499089093aafd7598eb5", "6976ec66c5944d5399e3ebf32d507589", "8b351b1d88004324beafb3a4ff1bbf13", "41bc45a943b0462d98ec845cb5011d7b", "41d6ab0d3b684019b49dedde4805d829", "864fd6c9b5df4487b6f5f7e46a1b9b4c", "dcc73886c9a7475fada1d1661c6e7fd9", "3d658012e1bd440eaad4d64383c60450", "dcd5ecd3aba143eeaebe54634b8ad2ff", "6e4c1874da8f494ca54114f5f09eac01", "b655197083314798b7ae8fa4643078bc", "e8080ec29fec471a854f0f2d0cbd42bf", "cc34d065576e48afbce25d3712dfc770", "9276f08af97e48fe87f0cd699ef3ccbc", "260a14186ae64251888d57a8d59bab14", "ab22fa2b2397475aa41f995794897d5e", "23a6ecd7a1cc4f6f8b06b58ef59399b2", "edfbbee6cbb7415a9c173273dc1ec722", "5635e1a9447648a5a16119465cc69e35", "111a21ea5a064acd9ce07522f3c86d20", "4593ece2846047fb823d598903e45c56" ] }, "id": "ed0OpduhX2JF", "outputId": "887d1454-060c-48df-95e3-4bc2e090fc39" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "VBox(children=(HTML(value='
Step | \n", "Training Loss | \n", "Validation Loss | \n", "
---|---|---|
10 | \n", "No log | \n", "0.433289 | \n", "
20 | \n", "No log | \n", "0.428100 | \n", "
30 | \n", "0.141000 | \n", "0.435952 | \n", "
40 | \n", "0.141000 | \n", "0.450911 | \n", "
50 | \n", "0.172400 | \n", "0.443676 | \n", "
"
]
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:91: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/bitsandbytes/autograd/_functions.py:316: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization\n",
" warnings.warn(f\"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization\")\n",
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:91: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/bitsandbytes/autograd/_functions.py:316: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization\n",
" warnings.warn(f\"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization\")\n",
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:91: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/bitsandbytes/autograd/_functions.py:316: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization\n",
" warnings.warn(f\"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization\")\n",
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:464: UserWarning: torch.utils.checkpoint: the use_reentrant parameter should be passed explicitly. In version 2.4 we will raise an exception if use_reentrant is not passed. use_reentrant=False is recommended, but if you need to preserve the current default behavior, you can pass use_reentrant=True. Refer to docs for more details on the differences between the two variants.\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/torch/utils/checkpoint.py:91: UserWarning: None of the inputs have requires_grad=True. Gradients will be None\n",
" warnings.warn(\n",
"/usr/local/lib/python3.10/dist-packages/bitsandbytes/autograd/_functions.py:316: UserWarning: MatMul8bitLt: inputs will be cast from torch.float32 to float16 during quantization\n",
" warnings.warn(f\"MatMul8bitLt: inputs will be cast from {A.dtype} to float16 during quantization\")\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"TrainOutput(global_step=50, training_loss=0.15669580936431884, metrics={'train_runtime': 1407.1261, 'train_samples_per_second': 0.569, 'train_steps_per_second': 0.036, 'total_flos': 1.71665620992e+18, 'train_loss': 0.15669580936431884, 'epoch': 0.13333333333333333})"
]
},
"metadata": {},
"execution_count": 35
}
],
"source": [
"trainer.train()"
],
"id": "ee8b7b8e-1c9a-4d77-9137-1778a629e6de"
},
{
"cell_type": "code",
"source": [
"\n",
"trainer.push_to_hub('End')"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 71
},
"id": "lf7oCLJAK0kv",
"outputId": "fde40f3d-89e3-4d6c-d1bb-24e7d7d62bc1"
},
"id": "lf7oCLJAK0kv",
"execution_count": 37,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"CommitInfo(commit_url='https://huggingface.co/pfe24/arab/commit/c80742daa5c202841d1baa48f3749eaa4b13cb5e', commit_message='End', commit_description='', oid='c80742daa5c202841d1baa48f3749eaa4b13cb5e', pr_url=None, pr_revision=None, pr_num=None)"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {},
"execution_count": 37
}
]
},
{
"cell_type": "code",
"source": [
"from peft import PeftModel, PeftConfig\n",
"from transformers import WhisperForConditionalGeneration, Seq2SeqTrainer\n",
"\n",
"peft_model_id = \"pfe24/arab\" # Use the same model ID as before.\n",
"peft_config = PeftConfig.from_pretrained(peft_model_id)\n",
"model = WhisperForConditionalGeneration.from_pretrained(\n",
" peft_config.base_model_name_or_path, load_in_8bit=True, device_map=\"auto\"\n",
")\n",
"model = PeftModel.from_pretrained(model, peft_model_id)\n",
"model.config.use_cache = True"
],
"metadata": {
"id": "gPpx9NsIw6UB",
"outputId": "d4e334bc-a5d0-43f4-8c78-0f9c0c19b346",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 751
}
},
"id": "gPpx9NsIw6UB",
"execution_count": 40,
"outputs": [
{
"output_type": "error",
"ename": "OSError",
"evalue": "None is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'\nIf this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=
Copy a token from your Hugging Face\ntokens page and paste it below.
Immediately click login after copying\nyour token or it might be stored in plain text in this notebook file.