| [[36m09-27 02:38:55[0m][[34mdora.distrib[0m][[32mINFO[0m] - world_size is 1, skipping init.[0m | |
| [[36m09-27 02:38:55[0m][[34mflashy.solver[0m][[32mINFO[0m] - Instantiating solver MusicGenSolver for XP 7bb1d622[0m | |
| [[36m09-27 02:38:55[0m][[34mflashy.solver[0m][[32mINFO[0m] - All XP logs are stored in /tmp/audiocraft_hari/xps/7bb1d622[0m | |
| [[36m09-27 02:38:55[0m][[34maudiocraft.solvers.builders[0m][[32mINFO[0m] - Loading audio data split train: /kaggle/working/audiocraft/egs/folk[0m | |
| [[36m09-27 02:38:55[0m][[34maudiocraft.solvers.builders[0m][[32mINFO[0m] - Loading audio data split valid: /kaggle/working/audiocraft/egs/folk[0m | |
| [[36m09-27 02:38:55[0m][[34maudiocraft.solvers.builders[0m][[32mINFO[0m] - Loading audio data split evaluate: /kaggle/working/audiocraft/egs/folk[0m | |
| [[36m09-27 02:38:55[0m][[34maudiocraft.solvers.builders[0m][[32mINFO[0m] - Loading audio data split generate: /kaggle/working/audiocraft/egs/folk[0m | |
| [[36m09-27 02:38:55[0m][[34mroot[0m][[32mINFO[0m] - Getting pretrained compression model from HF facebook/encodec_32khz[0m | |
| [[36m09-27 02:38:55[0m][[34mflashy.solver[0m][[32mINFO[0m] - Compression model has 4 codebooks with 2048 cardinality, and a framerate of 50[0m | |
| [[36m09-27 02:38:55[0m][[34maudiocraft.modules.conditioners[0m][[32mINFO[0m] - T5 will be evaluated with autocast as float32[0m | |
| [[36m09-27 02:39:01[0m][[34mflashy.solver[0m][[32mINFO[0m] - Model hash: 84d640e215de7863e944e465549d3e2e5faa07eb[0m | |
| [[36m09-27 02:39:01[0m][[34mflashy.solver[0m][[32mINFO[0m] - Initializing EMA on the model with decay = 0.99 every 10 updates[0m | |
| [[36m09-27 02:39:01[0m][[34mflashy.solver[0m][[32mINFO[0m] - Model size: 420.37 M params[0m | |
| [[36m09-27 02:39:01[0m][[34mflashy.solver[0m][[32mINFO[0m] - Base memory usage, with model, grad and optim: 6.73 GB[0m | |
| [[36m09-27 02:39:01[0m][[34mflashy.solver[0m][[32mINFO[0m] - Restoring weights and history.[0m | |
| [[36m09-27 02:39:01[0m][[34mflashy.solver[0m][[32mINFO[0m] - Loading a pretrained model. Ignoring 'load_best' and 'ignore_state_keys' params.[0m | |
| [[36m09-27 02:39:02[0m][[34mflashy.solver[0m][[32mINFO[0m] - Checkpoint source is not the current xp: Load state_dict from best state.[0m | |
| [[36m09-27 02:39:02[0m][[34mflashy.solver[0m][[32mINFO[0m] - Ignoring keys when loading best [][0m | |
| [[36m09-27 02:39:02[0m][[34mflashy.solver[0m][[32mINFO[0m] - Loading state_dict from best state.[0m | |
| [[36m09-27 02:39:04[0m][[34mflashy.solver[0m][[32mINFO[0m] - Re-initializing EMA from best state[0m | |
| [[36m09-27 02:39:04[0m][[34mflashy.solver[0m][[32mINFO[0m] - Initializing EMA on the model with decay = 0.99 every 10 updates[0m | |
| [[36m09-27 02:39:06[0m][[34mflashy.solver[0m][[32mINFO[0m] - Model hash: 776d041cbbcb8973c4968782a79f9bb63b53a727[0m | |
| [[36m09-27 02:39:19[0m][[34maudiocraft.modules.codebooks_patterns[0m][[32mINFO[0m] - New pattern, time steps: 1500, sequence steps: 1504[0m | |
| [[36m09-27 02:41:14[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 100/1000 | 0.79 it/sec | lr 9.55E-05 | grad_norm INF | grad_scale 18979.485 | ce 3.950 | ppl 62.856[0m | |
| [[36m09-27 02:43:13[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 200/1000 | 0.82 it/sec | lr 9.99E-05 | grad_norm 7.454E+00 | grad_scale 16384.000 | ce 3.642 | ppl 44.336[0m | |
| [[36m09-27 02:45:14[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 300/1000 | 0.82 it/sec | lr 9.99E-05 | grad_norm 5.754E+00 | grad_scale 16384.000 | ce 3.589 | ppl 39.746[0m | |
| [[36m09-27 02:47:16[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 400/1000 | 0.82 it/sec | lr 9.97E-05 | grad_norm 5.457E+00 | grad_scale 16384.000 | ce 3.338 | ppl 32.399[0m | |
| [[36m09-27 02:49:17[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 500/1000 | 0.82 it/sec | lr 9.95E-05 | grad_norm 4.997E+00 | grad_scale 16384.000 | ce 3.328 | ppl 31.657[0m | |
| [[36m09-27 02:51:19[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 600/1000 | 0.82 it/sec | lr 9.93E-05 | grad_norm 4.529E+00 | grad_scale 16384.000 | ce 3.266 | ppl 29.985[0m | |
| [[36m09-27 02:53:21[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 700/1000 | 0.82 it/sec | lr 9.90E-05 | grad_norm 4.046E+00 | grad_scale 16384.000 | ce 3.105 | ppl 25.236[0m | |
| [[36m09-27 02:55:22[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 800/1000 | 0.82 it/sec | lr 9.86E-05 | grad_norm 3.947E+00 | grad_scale 16384.000 | ce 3.092 | ppl 25.148[0m | |
| [[36m09-27 02:57:24[0m][[34mflashy.solver[0m][[32mINFO[0m] - Train | Epoch 1 | 900/1000 | 0.82 it/sec | lr 9.83E-05 | grad_norm 3.829E+00 | grad_scale 16384.000 | ce 2.974 | ppl 23.287[0m | |
| [[36m09-27 02:59:25[0m][[34mflashy.solver[0m][[32mINFO[0m] - [1mTrain Summary | Epoch 1 | lr=9.88E-05 | grad_norm=INF | grad_scale=16646.144 | ce=3.343 | ppl=33.996 | duration=1218.702[0m | |
| [[36m09-27 02:59:38[0m][[34mflashy.solver[0m][[32mINFO[0m] - [1mValid Summary | Epoch 1 | ce=2.382 | ppl=10.829 | duration=12.130[0m | |
| [ |