File size: 17,090 Bytes
434004a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 |
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<div align=\"center\">\n",
" <img src=\"https://user-images.githubusercontent.com/11793384/215372703-4385f66a-abe4-44c7-9626-96b7b65270c8.png\" width=\"40%\" height=\"40%\" />\n",
"</div>\n",
"\n",
"<center><font face=\"微软雅黑\" size=5> <b>【飞桨黑客松-AIGC - DreamBooth LoRA】This is my HomeTown</b></font></center> \n",
"\n",
"<br>\n",
"\n",
"<center>🔥🔥🔥 前往官网了解题目详情、报名、冲大奖、玩创意！👉👉👉 <a href=\"https://github.com/PaddlePaddle/Paddle/issues/50631#task105\">https://github.com/PaddlePaddle/Paddle/issues/50631#task105</a> </center>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 1. 背景介绍\n",
"\n",
"<h2> <center>紫金花海</center>\n",
" <br>\n",
" <center>The Purple Flower Sea</center> \n",
" <br>\n",
" <left> <font size=\"2\">紫金苑景区，金昌市民习惯称为“紫金花海”，位于金昌市区的北部，新华路街道天津路与泰安路交叉口北150米，总面积有700多亩。是按照景城一体化理念打造的香草花卉生态休闲旅游景区。</font></left>\n",
"</h2>\n",
"\n",
"\n",
"\n",
"<center> \n",
" <font size=\"2\">\n",
" \n",
" 微风袭来\n",
" 荡漾起的紫色“海浪”映入眼帘\n",
" 沁人心脾的清香\n",
" 淡淡地飘过每个角落\n",
" 小憩在花海旁\n",
" 倾听着花的物语\n",
" 骑着单车\n",
" 穿梭在紫色海洋\n",
" 掠过小湖\n",
" 静静享受紫色连波的浪漫\n",
" 走过小桥\n",
" 感受着紫色田园的美好\n",
"\n",
"\n",
" \n",
" <center>\n",
" <font size=\"2\">\n",
" \n",
" 被花海围绕的湖边\n",
" 倒映出一张张欢乐的笑脸\n",
" 穿过小小的别墅庄园\n",
" 来到充满爱意的婚礼广场\n",
" 浪漫而惬意的气氛\n",
" 像是走进了童话世界\n",
" 蜜蜂在“海浪”中辛勤游荡\n",
" 蝴蝶在“海浪”中嬉戏打闹\n",
" 小鸟在“海浪”中合奏吟唱\n",
" 人儿在“海浪”中忘记烦恼\n",
" 一个个穿梭在“海浪”的身影\n",
" 仿佛都在诉说着这别致的一切\n",
"\n",
" \n",
" \n",
" <center> \n",
" <font size=\"2\">\n",
" \n",
" 迎着初秋的微风\n",
" 远离城市的喧嚣\n",
" 游荡在这梦幻浪漫的紫色海洋\n",
" 把一切的不开心都遗忘\n",
" 欢迎大家来紫色小城游玩噢~\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 2. 准备工作 \n",
"## 2.1 环境安装 \n",
" \n",
"```sh\n",
"pip install \"paddlenlp>=2.5.2\" \"ppdiffusers>=0.11.1\" safetensors --user\n",
"```\n",
"</font>\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [],
"source": [
"# 请运行这里安装所需要的依赖环境！！\n",
"!pip install \"paddlenlp==2.5.2\" safetensors \"ppdiffusers>=0.11.1\" --user"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2.2 [Hugging Face Space](https://huggingface.co/) 注册和登录 \n",
"\n",
"题目要求将模型上传到 Hugging Face，需要先注册、登录。\n",
"\n",
"* <font face=\"微软雅黑\" size=3>**注册和登录：https://huggingface.co/join** </font>\n",
" <div align=\"center\">\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/fa96268f8d0047fa8ad25b678fb209631ebe08c463884378bdd268fdc025a808\" width=\"40%\" height=\"40%\" />\n",
" </div> \n",
"* <font face=\"微软雅黑\" size=3>**获取登录 Token**</font>\n",
" <div align=\"center\">\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/5cd101321369427ca94d685ad082779084243fe6839f49bc8b173478a8eaf03c\" width=\"40%\" height=\"40%\" />\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/7df3372db6f94ba2bdb636ee4a23c74bf2e05ef5a6e849c4b2e5fb3d4e3679e1\" width=\"40%\" height=\"40%\" />\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/ebc309852fba4ae6b799d519d196b5eee76bf21682db4cb8852e01d6ae6f0602\" width=\"40%\" height=\"40%\" /> \n",
" </div> \n",
"\n",
"* <font face=\"微软雅黑\" size=3>**Aistudio 登录 Huggingface Hub**</font>\n",
" <div align=\"center\">\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/d9e2c88b9a8d43f08f7379f4c8bef35c81992475e98b4c958f337f02b8174b56\" width=\"40%\" height=\"40%\" />\n",
" </div> \n",
" \n",
"Tips：为了方便我们之后上传权重，我们需要登录 Huggingface Hub，想要了解更多的信息我们可以查阅 [官方文档](https://huggingface.co/docs/huggingface_hub/quick-start)。\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"execution": {
"iopub.execute_input": "2023-03-11T05:39:13.575730Z",
"iopub.status.busy": "2023-03-11T05:39:13.575259Z",
"iopub.status.idle": "2023-03-11T05:39:14.194481Z",
"shell.execute_reply": "2023-03-11T05:39:14.193575Z",
"shell.execute_reply.started": "2023-03-11T05:39:13.575699Z"
},
"scrolled": true,
"tags": []
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "b5d6b75aa009457784dcba9d1597d9c3",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"VBox(children=(HTML(value='<center> <img\\nsrc=https://huggingface.co/front/assets/huggingface_logo-noborder.sv…"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# 登录 Huggingface Hub\n",
"!git config --global credential.helper store\n",
"from huggingface_hub import login\n",
"login()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- tips：如何检测是否登录成功？\n",
"\n",
"打开日志控制控制台，查看日志。\n",
"\n",
"\n",
"\n",
"登录成功时，日志如下：\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3. 如何训练模型，并上传到HF\n",
"\n",
"## 3.1 上传图片\n",
"* <font face=\"微软雅黑\" size=3>首先，我们需要将所需训练的图片上传到aistudio上的文件夹, 我们可以通过👉**拖拽上传** 的方式，将我们所需的图片上传至指定的文件夹。</font>\n",
"* <font face=\"微软雅黑\" size=3>在这里，我们已经在👉**hometown**文件夹准备好了如下所示的5张图片。</font>\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.2 训练参数调整\n",
"\n",
"<font face=\"微软雅黑\" size=3> 在训练过程中，我们可以尝试修改训练的默认参数，下面将从三个方面介绍部分参数。\n",
"\n",
"👉主要修改的参数：\n",
"> * **pretrained_model_name_or_path** ：想要训练的模型名称或者本地路径的模型，例如：`\"runwayml/stable-diffusion-v1-5\"`，更多模型可参考 [PaddleNLP 文档](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/ppdiffusers#ppdiffusers%E6%A8%A1%E5%9E%8B%E6%94%AF%E6%8C%81%E7%9A%84%E6%9D%83%E9%87%8D)。\n",
"> * **instance_data_dir**：训练图片所在的文件夹目录，我们可以将图片上传至aistudio项目。\n",
"> * **instance_prompt**：训练所使用的 `Prompt` 文本。\n",
"> * **resolution**：训练时图像的分辨率，建议为 `512`。\n",
"> * **output_dir**：训练过程中，模型保存的目录。\n",
"> * **checkpointing_steps**：每隔多少步保存模型，默认为`100`步。\n",
"> * **learning_rate**：训练使用的学习率，当我使用 `LoRA` 训练模型的时候，我们需要使用更大的学习率，因此我们这里使用 `1e-4` 而不是 `2e-6`。\n",
"> * **max_train_steps**：最大训练的步数，默认为`500`步。\n",
" \n",
"👉可选修改的参数：\n",
"> * **train_batch_size**：训练时候使用的 `batch_size`，当我们的GPU显存比较大的时候可以加大这个值，默认值为`4`。\n",
"> * **gradient_accumulation_steps**：梯度累积的步数，当我们GPU显存比较小的时候还想模拟大的训练批次，我们可以适当增加梯度累积的步数，默认值为`1`。\n",
"> * **seed**：随机种子，设置后可以复现训练结果。\n",
"> * **lora_rank**：`LoRA` 层的 `rank` 值，默认值为4，最终我们会得到 **3.5MB** 的模型，我们可以适当修改这个值，如：`32、64、128、256` 等。\n",
"> * **lr_scheduler**：学习率衰减策略，可以是`\"linear\", \"constant\", \"cosine\"`等。\n",
"> * **lr_warmup_steps**：学习率衰减前，`warmup` 到最大学习率所需要的步数。\n",
" \n",
"👉训练过程中评估使用的参数：\n",
"> * **num_validation_images**：训练的过程中，我们希望返回多少张图片，默认值为`4`张图片。\n",
"> * **validation_prompt**：训练的过程中我们会评估训练的怎么样，因此我们需要设置评估使用的 `prompt` 文本。\n",
"> * **validation_steps**：每隔多少个 `steps` 评估模型，我们可以查看训练的进度条，知道当前到了第几个 `steps`。\n",
" \n",
"🔥**Tips:**\n",
" 训练过程中会每隔 `validation_steps` 将生成的图片保存到 `{你指定的输出路径}/validation_images/{步数}.jpg`\n",
"\n",
"<div align=\"center\">\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/86ab5ed4a31f4cf1a4b9ba1c52ccc94e2731c711dff6420eb3745f718431cb29\" width=\"40%\" height=\"40%\" />\n",
"</div> \n",
" \n",
"👉权重上传的参数：\n",
"> * **push_to_hub**: 是否将模型上传到 `huggingface hub`，默认值为 `False`。\n",
"> * **hub_token**: 上传到 `huggingface hub` 所需要使用的 `token`，如果我们已经登录了，那么我们就无需填写。\n",
"> * **hub_model_id**: 上传到 `huggingface hub` 的模型库名称， 如果为 `None` 的话表示我们将使用 `output_dir` 的名称作为模型库名称。\n",
"\n",
"在下面的例子中，由于我们前面已经登录了，因此我们可以开启 **push_to_hub** 按钮，将最终训练好的模型同步上传到 `huggingface.co`\n",
" \n",
"当我们开启`push_to_hub`后，等待程序运行完毕后会自动将权重上传到这个路径 https://huggingface.co/{你的用户名}/{你指定的输出路径} ，例如： https://huggingface.co/peteli/hometown\n",
"</font>\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [],
"source": [
"!python train_dreambooth_lora.py \\\n",
" --pretrained_model_name_or_path=\"runwayml/stable-diffusion-v1-5\" \\\n",
" --instance_data_dir=\"./hometown\" \\\n",
" --output_dir=\"lora_outputs\" \\\n",
" --instance_prompt=\"a sea of purple and gold flowers\" \\\n",
" --resolution=512 \\\n",
" --train_batch_size=1 \\\n",
" --gradient_accumulation_steps=1 \\\n",
" --checkpointing_steps=100 \\\n",
" --learning_rate=1e-4 \\\n",
" --lr_scheduler=\"constant\" \\\n",
" --lr_warmup_steps=0 \\\n",
" --max_train_steps=1000 \\\n",
" --seed=0 \\\n",
" --lora_rank=4 \\\n",
" --push_to_hub=False \\\n",
" --validation_prompt=\"a sea of lavender and gold flowers in the world of fairy tales is my hometown\" \\\n",
" --validation_steps=100 \\\n",
" --num_validation_images=4"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.3 可视化训练过程\n",
"\n",
"VisualDL使用参考：[官方教程](https://aistudio.baidu.com/aistudio/projectdetail/1990920)\n",
"\n",
"我们可以参照如图所示的步骤，开启visualdl，然后查看训练过程中的指标变化。\n",
"\n",
"<div align=\"center\">\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/284485f03165482493e8399c3277a422ec5520382c564e2b9e1d5c0e113588e1\" width=\"60%\" height=\"60%\" />\n",
"</div>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.4 挑选满意的权重上传至Huggingface\n",
"\n",
"参数解释：\n",
"> * **upload_dir**：我们需要上传的文件夹目录。\n",
"> * **repo_name**：我们需要上传的repo名称，最终我们会上传到 https://huggingface.co/{你的用户名}/{你指定的repo名称}， 例如： https://huggingface.co/peteli/hometown.\n",
"> * **pretrained_model_name_or_path**：训练该模型所使用的基础模型。\n",
"> * **prompt**：搭配该权重需要使用的Prompt文本。\n",
"<div align=\"center\">\n",
" <img src=\"https://ai-studio-static-online.cdn.bcebos.com/9036aa64b12d4d1db0b64f685aa98c600aad8f984cbf4ef592bef4c4e6db11e4\" width=\"60%\" height=\"60%\" />\n",
"</div>"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [],
"source": [
"from utils import upload_lora_folder\n",
"upload_dir = \"lora_outputs\" # 我们需要上传的文件夹目录\n",
"repo_name = \"hometown\" # 我们需要上传的repo名称\n",
"pretrained_model_name_or_path = \"runwayml/stable-diffusion-v1-5\" # 训练该模型所使用的基础模型\n",
"prompt = \"a sea of lavender and gold flowers in the world of fairy tales is my hometown\" # 搭配该权重需要使用的Prompt文本\n",
"\n",
"upload_lora_folder(\n",
" upload_dir=upload_dir,\n",
" repo_name=repo_name,\n",
" pretrained_model_name_or_path=pretrained_model_name_or_path,\n",
" prompt=prompt, \n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 4.效果展示\n",
"\n",
"#### 图1：\n",
"\n",
"\n",
"\n",
"#### 图2：\n",
"\n",
"\n",
"\n",
"#### 图3：\n",
"\n",
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "py35-paddle1.2.0"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
},
"toc-autonumbering": false,
"toc-showcode": false,
"toc-showtags": false
},
"nbformat": 4,
"nbformat_minor": 4
}
|