2024-02-22 09:45:43,037 INFO [train.py:734] (2/4) Training started
2024-02-22 09:45:43,038 INFO [train.py:744] (2/4) Device: cuda:2
2024-02-22 09:45:43,039 INFO [train.py:750] (2/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': -1, 'log_interval': 50, 'valid_interval': 200, 'env_info': {'k2-version': '1.24.4', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '44a9d5682af9fd3ef77074777e15278ec6d390eb', 'k2-git-date': 'Wed Sep 27 11:22:55 2023', 'lhotse-version': '0.0.0+unknown.version', 'torch-version': '1.10.0+cu102', 'torch-cuda-available': True, 'torch-cuda-version': '10.2', 'python-version': '3.8', 'icefall-git-branch': 'update-vits-tokenizer', 'icefall-git-sha1': '595d4a3c-clean', 'icefall-git-date': 'Wed Feb 21 17:50:19 2024', 'icefall-path': '/star-zw/workspace/tts/icefall_tts', 'k2-path': '/star-zw/workspace/k2/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-zw/workspace/lhotse/lhotse_dev/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-7-1218101249-5bcbfb5567-jsftr', 'IP address': '10.177.6.147'}, 'sampling_rate': 22050, 'frame_shift': 256, 'frame_length': 1024, 'feature_dim': 513, 'n_mels': 80, 'lambda_adv': 1.0, 'lambda_mel': 45.0, 'lambda_feat_match': 2.0, 'lambda_dur': 1.0, 'lambda_kl': 1.0, 'world_size': 4, 'master_port': 12354, 'tensorboard': True, 'num_epochs': 1000, 'start_epoch': 1, 'exp_dir': PosixPath('vits/exp-new-tokenizer-add-sos-eos-4-gpu'), 'tokens': 'data/tokens.txt', 'lr': 0.0002, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 20, 'use_fp16': True, 'manifest_dir': PosixPath('data/spectrogram'), 'max_duration': 500, 'bucketing_sampler': True, 'num_buckets': 30, 'on_the_fly_feats': False, 'shuffle': True, 'drop_last': True, 'return_cuts': False, 'num_workers': 2, 'input_strategy': 'PrecomputedFeatures', 'blank_id': 0, 'vocab_size': 159}
2024-02-22 09:45:43,039 INFO [train.py:752] (2/4) About to create model
2024-02-22 09:45:45,458 INFO [train.py:758] (2/4) Number of parameters in generator: 35637106
2024-02-22 09:45:45,458 INFO [train.py:760] (2/4) Number of parameters in discriminator: 50974956
2024-02-22 09:45:45,459 INFO [train.py:761] (2/4) Total number of parameters: 86612062
2024-02-22 09:45:50,196 INFO [train.py:768] (2/4) Using DDP
2024-02-22 09:45:50,808 INFO [tts_datamodule.py:310] (2/4) About to get train cuts
2024-02-22 09:45:51,150 INFO [tts_datamodule.py:169] (2/4) About to create train dataset
2024-02-22 09:45:51,151 INFO [tts_datamodule.py:193] (2/4) Using DynamicBucketingSampler.
2024-02-22 09:45:52,666 INFO [tts_datamodule.py:210] (2/4) About to create train dataloader
2024-02-22 09:45:52,667 INFO [tts_datamodule.py:317] (2/4) About to get validation cuts
2024-02-22 09:45:52,668 INFO [tts_datamodule.py:233] (2/4) About to create dev dataset
2024-02-22 09:45:52,675 INFO [tts_datamodule.py:260] (2/4) About to create valid dataloader
2024-02-22 09:45:52,675 INFO [train.py:662] (2/4) Sanity check -- see if any of the batches in epoch 1 would cause OOM.
2024-02-22 09:46:00,121 INFO [train.py:709] (2/4) Maximum memory allocated so far is 15115MB
2024-02-22 09:46:04,007 INFO [train.py:709] (2/4) Maximum memory allocated so far is 15450MB
2024-02-22 09:46:10,345 INFO [train.py:709] (2/4) Maximum memory allocated so far is 16459MB
2024-02-22 09:46:14,776 INFO [train.py:709] (2/4) Maximum memory allocated so far is 16464MB
2024-02-22 09:46:25,851 INFO [train.py:709] (2/4) Maximum memory allocated so far is 26569MB
2024-02-22 09:46:34,155 INFO [train.py:709] (2/4) Maximum memory allocated so far is 26579MB
2024-02-22 09:46:34,182 INFO [train.py:845] (2/4) Start epoch 1
2024-02-22 09:46:46,784 INFO [train.py:471] (2/4) Epoch 1, batch 0, global_batch_idx: 0, batch size: 53, loss[discriminator_loss=6, discriminator_real_loss=6, discriminator_fake_loss=0.0009675, generator_loss=1053, generator_mel_loss=80.42, generator_kl_loss=966, generator_dur_loss=1.899, generator_adv_loss=4.754, generator_feat_match_loss=0.2651, over 53.00 samples.], tot_loss[discriminator_loss=6, discriminator_real_loss=6, discriminator_fake_loss=0.0009675, generator_loss=1053, generator_mel_loss=80.42, generator_kl_loss=966, generator_dur_loss=1.899, generator_adv_loss=4.754, generator_feat_match_loss=0.2651, over 53.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 2.0
2024-02-22 09:46:46,785 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 09:47:04,241 INFO [train.py:534] (2/4) Epoch 1, validation: discriminator_loss=4.831, discriminator_real_loss=4.75, discriminator_fake_loss=0.08159, generator_loss=531.1, generator_mel_loss=71.89, generator_kl_loss=452.3, generator_dur_loss=1.85, generator_adv_loss=4.756, generator_feat_match_loss=0.3225, over 100.00 samples.
2024-02-22 09:47:04,242 INFO [train.py:535] (2/4) Maximum memory allocated so far is 26579MB
2024-02-22 09:50:16,965 INFO [train.py:845] (2/4) Start epoch 2
2024-02-22 09:51:45,707 INFO [train.py:471] (2/4) Epoch 2, batch 13, global_batch_idx: 50, batch size: 63, loss[discriminator_loss=3.268, discriminator_real_loss=2.631, discriminator_fake_loss=0.6367, generator_loss=62.75, generator_mel_loss=43.05, generator_kl_loss=13.12, generator_dur_loss=1.636, generator_adv_loss=2.723, generator_feat_match_loss=2.223, over 63.00 samples.], tot_loss[discriminator_loss=2.694, discriminator_real_loss=1.646, discriminator_fake_loss=1.047, generator_loss=66.76, generator_mel_loss=43.91, generator_kl_loss=15.68, generator_dur_loss=1.564, generator_adv_loss=2.608, generator_feat_match_loss=2.992, over 1286.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 2.0
2024-02-22 09:53:43,534 INFO [train.py:845] (2/4) Start epoch 3
2024-02-22 09:56:04,456 INFO [train.py:471] (2/4) Epoch 3, batch 26, global_batch_idx: 100, batch size: 64, loss[discriminator_loss=3.141, discriminator_real_loss=1.959, discriminator_fake_loss=1.182, generator_loss=53.9, generator_mel_loss=39.62, generator_kl_loss=6.488, generator_dur_loss=1.648, generator_adv_loss=3.361, generator_feat_match_loss=2.781, over 64.00 samples.], tot_loss[discriminator_loss=2.823, discriminator_real_loss=1.591, discriminator_fake_loss=1.232, generator_loss=54.76, generator_mel_loss=40.18, generator_kl_loss=7.403, generator_dur_loss=1.647, generator_adv_loss=2.521, generator_feat_match_loss=3.011, over 1710.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 4.0
2024-02-22 09:57:09,523 INFO [train.py:845] (2/4) Start epoch 4
2024-02-22 10:00:39,555 INFO [train.py:845] (2/4) Start epoch 5
2024-02-22 10:01:06,422 INFO [train.py:471] (2/4) Epoch 5, batch 2, global_batch_idx: 150, batch size: 69, loss[discriminator_loss=3.117, discriminator_real_loss=1.82, discriminator_fake_loss=1.297, generator_loss=47.86, generator_mel_loss=38.06, generator_kl_loss=4.004, generator_dur_loss=1.651, generator_adv_loss=1.776, generator_feat_match_loss=2.367, over 69.00 samples.], tot_loss[discriminator_loss=3.448, discriminator_real_loss=2.186, discriminator_fake_loss=1.262, generator_loss=47.55, generator_mel_loss=37.73, generator_kl_loss=4.173, generator_dur_loss=1.653, generator_adv_loss=1.832, generator_feat_match_loss=2.167, over 265.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 4.0
2024-02-22 10:04:07,826 INFO [train.py:845] (2/4) Start epoch 6
2024-02-22 10:05:45,751 INFO [train.py:471] (2/4) Epoch 6, batch 15, global_batch_idx: 200, batch size: 60, loss[discriminator_loss=2.648, discriminator_real_loss=1.284, discriminator_fake_loss=1.363, generator_loss=45.91, generator_mel_loss=36.96, generator_kl_loss=3.188, generator_dur_loss=1.672, generator_adv_loss=1.869, generator_feat_match_loss=2.219, over 60.00 samples.], tot_loss[discriminator_loss=2.832, discriminator_real_loss=1.472, discriminator_fake_loss=1.36, generator_loss=45.48, generator_mel_loss=36.94, generator_kl_loss=3.309, generator_dur_loss=1.676, generator_adv_loss=1.802, generator_feat_match_loss=1.756, over 1150.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 8.0
2024-02-22 10:05:45,753 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 10:05:53,954 INFO [train.py:534] (2/4) Epoch 6, validation: discriminator_loss=2.526, discriminator_real_loss=1.288, discriminator_fake_loss=1.238, generator_loss=46.21, generator_mel_loss=37.16, generator_kl_loss=3.183, generator_dur_loss=1.709, generator_adv_loss=1.894, generator_feat_match_loss=2.263, over 100.00 samples.
2024-02-22 10:05:53,955 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28080MB
2024-02-22 10:07:46,253 INFO [train.py:845] (2/4) Start epoch 7
2024-02-22 10:10:29,707 INFO [train.py:471] (2/4) Epoch 7, batch 28, global_batch_idx: 250, batch size: 76, loss[discriminator_loss=2.916, discriminator_real_loss=1.356, discriminator_fake_loss=1.56, generator_loss=44.42, generator_mel_loss=35.84, generator_kl_loss=2.74, generator_dur_loss=1.694, generator_adv_loss=2.113, generator_feat_match_loss=2.029, over 76.00 samples.], tot_loss[discriminator_loss=2.659, discriminator_real_loss=1.445, discriminator_fake_loss=1.213, generator_loss=45.08, generator_mel_loss=36.07, generator_kl_loss=2.87, generator_dur_loss=1.674, generator_adv_loss=2.069, generator_feat_match_loss=2.395, over 2170.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 8.0
2024-02-22 10:11:12,855 INFO [train.py:845] (2/4) Start epoch 8
2024-02-22 10:14:37,962 INFO [train.py:845] (2/4) Start epoch 9
2024-02-22 10:15:14,748 INFO [train.py:471] (2/4) Epoch 9, batch 4, global_batch_idx: 300, batch size: 65, loss[discriminator_loss=2.516, discriminator_real_loss=1.451, discriminator_fake_loss=1.064, generator_loss=43.15, generator_mel_loss=34.93, generator_kl_loss=2.334, generator_dur_loss=1.711, generator_adv_loss=1.832, generator_feat_match_loss=2.336, over 65.00 samples.], tot_loss[discriminator_loss=2.639, discriminator_real_loss=1.329, discriminator_fake_loss=1.309, generator_loss=43.08, generator_mel_loss=34.9, generator_kl_loss=2.378, generator_dur_loss=1.671, generator_adv_loss=1.965, generator_feat_match_loss=2.166, over 370.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 8.0
2024-02-22 10:18:11,177 INFO [train.py:845] (2/4) Start epoch 10
2024-02-22 10:19:57,821 INFO [train.py:471] (2/4) Epoch 10, batch 17, global_batch_idx: 350, batch size: 67, loss[discriminator_loss=2.754, discriminator_real_loss=1.771, discriminator_fake_loss=0.9844, generator_loss=42.57, generator_mel_loss=34.64, generator_kl_loss=2.196, generator_dur_loss=1.708, generator_adv_loss=1.816, generator_feat_match_loss=2.207, over 67.00 samples.], tot_loss[discriminator_loss=2.686, discriminator_real_loss=1.433, discriminator_fake_loss=1.253, generator_loss=42.53, generator_mel_loss=34.56, generator_kl_loss=2.266, generator_dur_loss=1.685, generator_adv_loss=1.986, generator_feat_match_loss=2.034, over 1207.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 8.0
2024-02-22 10:21:40,198 INFO [train.py:845] (2/4) Start epoch 11
2024-02-22 10:24:31,437 INFO [train.py:471] (2/4) Epoch 11, batch 30, global_batch_idx: 400, batch size: 55, loss[discriminator_loss=2.543, discriminator_real_loss=1.136, discriminator_fake_loss=1.408, generator_loss=42.05, generator_mel_loss=33.35, generator_kl_loss=2.129, generator_dur_loss=1.709, generator_adv_loss=2.027, generator_feat_match_loss=2.828, over 55.00 samples.], tot_loss[discriminator_loss=2.444, discriminator_real_loss=1.356, discriminator_fake_loss=1.088, generator_loss=42.89, generator_mel_loss=33.86, generator_kl_loss=2.154, generator_dur_loss=1.69, generator_adv_loss=2.26, generator_feat_match_loss=2.923, over 2159.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 10:24:31,438 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 10:24:39,869 INFO [train.py:534] (2/4) Epoch 11, validation: discriminator_loss=2.268, discriminator_real_loss=1.115, discriminator_fake_loss=1.153, generator_loss=43.03, generator_mel_loss=33.87, generator_kl_loss=2.199, generator_dur_loss=1.704, generator_adv_loss=2.041, generator_feat_match_loss=3.221, over 100.00 samples.
2024-02-22 10:24:39,870 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28084MB
2024-02-22 10:25:16,402 INFO [train.py:845] (2/4) Start epoch 12
2024-02-22 10:28:45,717 INFO [train.py:845] (2/4) Start epoch 13
2024-02-22 10:29:32,816 INFO [train.py:471] (2/4) Epoch 13, batch 6, global_batch_idx: 450, batch size: 90, loss[discriminator_loss=2.785, discriminator_real_loss=1.519, discriminator_fake_loss=1.266, generator_loss=40.53, generator_mel_loss=32.8, generator_kl_loss=2.055, generator_dur_loss=1.664, generator_adv_loss=2.084, generator_feat_match_loss=1.931, over 90.00 samples.], tot_loss[discriminator_loss=2.611, discriminator_real_loss=1.386, discriminator_fake_loss=1.225, generator_loss=41.57, generator_mel_loss=33.28, generator_kl_loss=2.042, generator_dur_loss=1.677, generator_adv_loss=2.12, generator_feat_match_loss=2.45, over 510.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 10:32:18,350 INFO [train.py:845] (2/4) Start epoch 14
2024-02-22 10:34:05,443 INFO [train.py:471] (2/4) Epoch 14, batch 19, global_batch_idx: 500, batch size: 64, loss[discriminator_loss=3.049, discriminator_real_loss=1.518, discriminator_fake_loss=1.531, generator_loss=40.08, generator_mel_loss=33.22, generator_kl_loss=1.97, generator_dur_loss=1.677, generator_adv_loss=1.678, generator_feat_match_loss=1.535, over 64.00 samples.], tot_loss[discriminator_loss=2.701, discriminator_real_loss=1.418, discriminator_fake_loss=1.283, generator_loss=41.49, generator_mel_loss=33.25, generator_kl_loss=2.01, generator_dur_loss=1.689, generator_adv_loss=2.1, generator_feat_match_loss=2.44, over 1356.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 10:35:41,652 INFO [train.py:845] (2/4) Start epoch 15
2024-02-22 10:38:42,739 INFO [train.py:471] (2/4) Epoch 15, batch 32, global_batch_idx: 550, batch size: 153, loss[discriminator_loss=2.719, discriminator_real_loss=1.299, discriminator_fake_loss=1.419, generator_loss=40.29, generator_mel_loss=32.91, generator_kl_loss=2.045, generator_dur_loss=1.632, generator_adv_loss=1.709, generator_feat_match_loss=1.988, over 153.00 samples.], tot_loss[discriminator_loss=2.757, discriminator_real_loss=1.415, discriminator_fake_loss=1.341, generator_loss=40.41, generator_mel_loss=32.79, generator_kl_loss=2.063, generator_dur_loss=1.675, generator_adv_loss=1.884, generator_feat_match_loss=2, over 2529.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 10:39:06,432 INFO [train.py:845] (2/4) Start epoch 16
2024-02-22 10:42:35,434 INFO [train.py:845] (2/4) Start epoch 17
2024-02-22 10:43:38,004 INFO [train.py:471] (2/4) Epoch 17, batch 8, global_batch_idx: 600, batch size: 52, loss[discriminator_loss=2.688, discriminator_real_loss=1.486, discriminator_fake_loss=1.202, generator_loss=39.41, generator_mel_loss=31.83, generator_kl_loss=2.057, generator_dur_loss=1.689, generator_adv_loss=1.803, generator_feat_match_loss=2.027, over 52.00 samples.], tot_loss[discriminator_loss=2.803, discriminator_real_loss=1.451, discriminator_fake_loss=1.351, generator_loss=39.12, generator_mel_loss=31.64, generator_kl_loss=2.103, generator_dur_loss=1.681, generator_adv_loss=1.858, generator_feat_match_loss=1.833, over 698.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 10:43:38,005 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 10:43:45,980 INFO [train.py:534] (2/4) Epoch 17, validation: discriminator_loss=2.629, discriminator_real_loss=1.268, discriminator_fake_loss=1.361, generator_loss=41.05, generator_mel_loss=33.52, generator_kl_loss=2.181, generator_dur_loss=1.681, generator_adv_loss=1.665, generator_feat_match_loss=2.001, over 100.00 samples.
2024-02-22 10:43:45,981 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28084MB
2024-02-22 10:46:18,544 INFO [train.py:845] (2/4) Start epoch 18
2024-02-22 10:48:26,431 INFO [train.py:471] (2/4) Epoch 18, batch 21, global_batch_idx: 650, batch size: 58, loss[discriminator_loss=2.664, discriminator_real_loss=1.521, discriminator_fake_loss=1.142, generator_loss=38.76, generator_mel_loss=31.13, generator_kl_loss=1.997, generator_dur_loss=1.708, generator_adv_loss=1.901, generator_feat_match_loss=2.02, over 58.00 samples.], tot_loss[discriminator_loss=2.634, discriminator_real_loss=1.374, discriminator_fake_loss=1.26, generator_loss=39.43, generator_mel_loss=31.44, generator_kl_loss=2.046, generator_dur_loss=1.681, generator_adv_loss=2.023, generator_feat_match_loss=2.237, over 1520.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 10:49:52,439 INFO [train.py:845] (2/4) Start epoch 19
2024-02-22 10:53:05,144 INFO [train.py:471] (2/4) Epoch 19, batch 34, global_batch_idx: 700, batch size: 49, loss[discriminator_loss=2.535, discriminator_real_loss=1.176, discriminator_fake_loss=1.358, generator_loss=39.15, generator_mel_loss=30.76, generator_kl_loss=2.061, generator_dur_loss=1.735, generator_adv_loss=2.229, generator_feat_match_loss=2.367, over 49.00 samples.], tot_loss[discriminator_loss=2.647, discriminator_real_loss=1.4, discriminator_fake_loss=1.247, generator_loss=39.04, generator_mel_loss=31.03, generator_kl_loss=2.036, generator_dur_loss=1.691, generator_adv_loss=2.035, generator_feat_match_loss=2.244, over 2346.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 10:53:20,909 INFO [train.py:845] (2/4) Start epoch 20
2024-02-22 10:56:52,231 INFO [train.py:845] (2/4) Start epoch 21
2024-02-22 10:57:56,149 INFO [train.py:471] (2/4) Epoch 21, batch 10, global_batch_idx: 750, batch size: 51, loss[discriminator_loss=2.652, discriminator_real_loss=1.536, discriminator_fake_loss=1.117, generator_loss=39.13, generator_mel_loss=31.49, generator_kl_loss=2.053, generator_dur_loss=1.719, generator_adv_loss=1.92, generator_feat_match_loss=1.949, over 51.00 samples.], tot_loss[discriminator_loss=2.613, discriminator_real_loss=1.355, discriminator_fake_loss=1.258, generator_loss=38.98, generator_mel_loss=30.85, generator_kl_loss=2.016, generator_dur_loss=1.684, generator_adv_loss=2.028, generator_feat_match_loss=2.402, over 781.00 samples.], cur_lr_g: 2.00e-04, cur_lr_d: 2.00e-04, grad_scale: 16.0
2024-02-22 11:00:25,443 INFO [train.py:845] (2/4) Start epoch 22
2024-02-22 11:02:42,536 INFO [train.py:471] (2/4) Epoch 22, batch 23, global_batch_idx: 800, batch size: 76, loss[discriminator_loss=2.707, discriminator_real_loss=1.167, discriminator_fake_loss=1.541, generator_loss=38.98, generator_mel_loss=30.91, generator_kl_loss=1.961, generator_dur_loss=1.678, generator_adv_loss=2.277, generator_feat_match_loss=2.154, over 76.00 samples.], tot_loss[discriminator_loss=2.653, discriminator_real_loss=1.389, discriminator_fake_loss=1.264, generator_loss=38.89, generator_mel_loss=30.82, generator_kl_loss=1.994, generator_dur_loss=1.685, generator_adv_loss=2.025, generator_feat_match_loss=2.367, over 1709.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:02:42,537 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 11:02:51,247 INFO [train.py:534] (2/4) Epoch 22, validation: discriminator_loss=2.629, discriminator_real_loss=1.587, discriminator_fake_loss=1.042, generator_loss=39.72, generator_mel_loss=31.3, generator_kl_loss=2.193, generator_dur_loss=1.675, generator_adv_loss=2.275, generator_feat_match_loss=2.276, over 100.00 samples.
2024-02-22 11:02:51,247 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28084MB
2024-02-22 11:04:02,875 INFO [train.py:845] (2/4) Start epoch 23
2024-02-22 11:07:36,609 INFO [train.py:471] (2/4) Epoch 23, batch 36, global_batch_idx: 850, batch size: 52, loss[discriminator_loss=2.531, discriminator_real_loss=1.246, discriminator_fake_loss=1.284, generator_loss=37.47, generator_mel_loss=29.56, generator_kl_loss=2.032, generator_dur_loss=1.69, generator_adv_loss=1.97, generator_feat_match_loss=2.217, over 52.00 samples.], tot_loss[discriminator_loss=2.676, discriminator_real_loss=1.43, discriminator_fake_loss=1.247, generator_loss=38.56, generator_mel_loss=30.66, generator_kl_loss=1.99, generator_dur_loss=1.675, generator_adv_loss=2.004, generator_feat_match_loss=2.237, over 2703.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:07:37,038 INFO [train.py:845] (2/4) Start epoch 24
2024-02-22 11:10:59,684 INFO [train.py:845] (2/4) Start epoch 25
2024-02-22 11:12:17,889 INFO [train.py:471] (2/4) Epoch 25, batch 12, global_batch_idx: 900, batch size: 56, loss[discriminator_loss=2.586, discriminator_real_loss=1.251, discriminator_fake_loss=1.334, generator_loss=38.77, generator_mel_loss=30.47, generator_kl_loss=1.91, generator_dur_loss=1.712, generator_adv_loss=2.215, generator_feat_match_loss=2.463, over 56.00 samples.], tot_loss[discriminator_loss=2.637, discriminator_real_loss=1.425, discriminator_fake_loss=1.212, generator_loss=39, generator_mel_loss=30.76, generator_kl_loss=1.969, generator_dur_loss=1.692, generator_adv_loss=2.077, generator_feat_match_loss=2.495, over 914.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:14:24,538 INFO [train.py:845] (2/4) Start epoch 26
2024-02-22 11:16:51,702 INFO [train.py:471] (2/4) Epoch 26, batch 25, global_batch_idx: 950, batch size: 69, loss[discriminator_loss=2.795, discriminator_real_loss=1.579, discriminator_fake_loss=1.216, generator_loss=37.19, generator_mel_loss=29.53, generator_kl_loss=1.911, generator_dur_loss=1.679, generator_adv_loss=2.029, generator_feat_match_loss=2.047, over 69.00 samples.], tot_loss[discriminator_loss=2.663, discriminator_real_loss=1.401, discriminator_fake_loss=1.262, generator_loss=38.04, generator_mel_loss=30.02, generator_kl_loss=1.948, generator_dur_loss=1.685, generator_adv_loss=2.03, generator_feat_match_loss=2.354, over 1845.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:17:55,496 INFO [train.py:845] (2/4) Start epoch 27
2024-02-22 11:21:23,425 INFO [train.py:845] (2/4) Start epoch 28
2024-02-22 11:21:41,629 INFO [train.py:471] (2/4) Epoch 28, batch 1, global_batch_idx: 1000, batch size: 110, loss[discriminator_loss=2.744, discriminator_real_loss=1.381, discriminator_fake_loss=1.363, generator_loss=37.94, generator_mel_loss=30.05, generator_kl_loss=1.995, generator_dur_loss=1.68, generator_adv_loss=2.086, generator_feat_match_loss=2.129, over 110.00 samples.], tot_loss[discriminator_loss=2.753, discriminator_real_loss=1.41, discriminator_fake_loss=1.342, generator_loss=37.9, generator_mel_loss=30.01, generator_kl_loss=2, generator_dur_loss=1.692, generator_adv_loss=2.016, generator_feat_match_loss=2.174, over 165.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:21:41,629 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 11:21:50,648 INFO [train.py:534] (2/4) Epoch 28, validation: discriminator_loss=2.567, discriminator_real_loss=1.44, discriminator_fake_loss=1.127, generator_loss=39.1, generator_mel_loss=30.63, generator_kl_loss=2.251, generator_dur_loss=1.683, generator_adv_loss=2.122, generator_feat_match_loss=2.423, over 100.00 samples.
2024-02-22 11:21:50,649 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28084MB
2024-02-22 11:25:05,313 INFO [train.py:845] (2/4) Start epoch 29
2024-02-22 11:26:33,682 INFO [train.py:471] (2/4) Epoch 29, batch 14, global_batch_idx: 1050, batch size: 52, loss[discriminator_loss=2.652, discriminator_real_loss=1.495, discriminator_fake_loss=1.158, generator_loss=37.62, generator_mel_loss=29.83, generator_kl_loss=1.978, generator_dur_loss=1.692, generator_adv_loss=1.845, generator_feat_match_loss=2.271, over 52.00 samples.], tot_loss[discriminator_loss=2.621, discriminator_real_loss=1.38, discriminator_fake_loss=1.241, generator_loss=37.93, generator_mel_loss=29.8, generator_kl_loss=1.943, generator_dur_loss=1.687, generator_adv_loss=2.075, generator_feat_match_loss=2.427, over 1171.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:28:32,626 INFO [train.py:845] (2/4) Start epoch 30
2024-02-22 11:31:07,098 INFO [train.py:471] (2/4) Epoch 30, batch 27, global_batch_idx: 1100, batch size: 69, loss[discriminator_loss=2.598, discriminator_real_loss=1.395, discriminator_fake_loss=1.204, generator_loss=36.97, generator_mel_loss=28.67, generator_kl_loss=1.92, generator_dur_loss=1.685, generator_adv_loss=2.092, generator_feat_match_loss=2.604, over 69.00 samples.], tot_loss[discriminator_loss=2.64, discriminator_real_loss=1.383, discriminator_fake_loss=1.257, generator_loss=37.74, generator_mel_loss=29.59, generator_kl_loss=1.948, generator_dur_loss=1.682, generator_adv_loss=2.043, generator_feat_match_loss=2.472, over 1966.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:31:57,516 INFO [train.py:845] (2/4) Start epoch 31
2024-02-22 11:35:22,953 INFO [train.py:845] (2/4) Start epoch 32
2024-02-22 11:35:55,593 INFO [train.py:471] (2/4) Epoch 32, batch 3, global_batch_idx: 1150, batch size: 95, loss[discriminator_loss=2.553, discriminator_real_loss=1.193, discriminator_fake_loss=1.359, generator_loss=38.03, generator_mel_loss=29.97, generator_kl_loss=1.892, generator_dur_loss=1.648, generator_adv_loss=1.992, generator_feat_match_loss=2.521, over 95.00 samples.], tot_loss[discriminator_loss=2.614, discriminator_real_loss=1.396, discriminator_fake_loss=1.218, generator_loss=37.43, generator_mel_loss=29.31, generator_kl_loss=1.939, generator_dur_loss=1.675, generator_adv_loss=2.086, generator_feat_match_loss=2.418, over 313.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:38:46,526 INFO [train.py:845] (2/4) Start epoch 33
2024-02-22 11:40:33,042 INFO [train.py:471] (2/4) Epoch 33, batch 16, global_batch_idx: 1200, batch size: 49, loss[discriminator_loss=2.578, discriminator_real_loss=1.328, discriminator_fake_loss=1.25, generator_loss=36.87, generator_mel_loss=28.87, generator_kl_loss=1.899, generator_dur_loss=1.727, generator_adv_loss=1.845, generator_feat_match_loss=2.531, over 49.00 samples.], tot_loss[discriminator_loss=2.638, discriminator_real_loss=1.412, discriminator_fake_loss=1.226, generator_loss=37.29, generator_mel_loss=29.07, generator_kl_loss=1.982, generator_dur_loss=1.673, generator_adv_loss=2.078, generator_feat_match_loss=2.482, over 1343.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:40:33,044 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 11:40:42,364 INFO [train.py:534] (2/4) Epoch 33, validation: discriminator_loss=2.646, discriminator_real_loss=1.216, discriminator_fake_loss=1.43, generator_loss=38.43, generator_mel_loss=30.22, generator_kl_loss=2.239, generator_dur_loss=1.714, generator_adv_loss=1.686, generator_feat_match_loss=2.58, over 100.00 samples.
2024-02-22 11:40:42,365 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28084MB
2024-02-22 11:42:22,852 INFO [train.py:845] (2/4) Start epoch 34
2024-02-22 11:45:09,196 INFO [train.py:471] (2/4) Epoch 34, batch 29, global_batch_idx: 1250, batch size: 153, loss[discriminator_loss=2.512, discriminator_real_loss=1.118, discriminator_fake_loss=1.394, generator_loss=37.66, generator_mel_loss=29.25, generator_kl_loss=2.064, generator_dur_loss=1.64, generator_adv_loss=1.949, generator_feat_match_loss=2.758, over 153.00 samples.], tot_loss[discriminator_loss=2.628, discriminator_real_loss=1.372, discriminator_fake_loss=1.256, generator_loss=37.1, generator_mel_loss=28.94, generator_kl_loss=2.009, generator_dur_loss=1.69, generator_adv_loss=2.021, generator_feat_match_loss=2.444, over 2207.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:45:47,523 INFO [train.py:845] (2/4) Start epoch 35
2024-02-22 11:49:18,710 INFO [train.py:845] (2/4) Start epoch 36
2024-02-22 11:49:56,855 INFO [train.py:471] (2/4) Epoch 36, batch 5, global_batch_idx: 1300, batch size: 52, loss[discriminator_loss=2.59, discriminator_real_loss=1.59, discriminator_fake_loss=1.001, generator_loss=37.49, generator_mel_loss=29.32, generator_kl_loss=2.013, generator_dur_loss=1.723, generator_adv_loss=1.977, generator_feat_match_loss=2.453, over 52.00 samples.], tot_loss[discriminator_loss=2.598, discriminator_real_loss=1.382, discriminator_fake_loss=1.216, generator_loss=37.18, generator_mel_loss=28.89, generator_kl_loss=1.984, generator_dur_loss=1.718, generator_adv_loss=2.012, generator_feat_match_loss=2.581, over 350.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:52:48,593 INFO [train.py:845] (2/4) Start epoch 37
2024-02-22 11:54:35,972 INFO [train.py:471] (2/4) Epoch 37, batch 18, global_batch_idx: 1350, batch size: 126, loss[discriminator_loss=2.539, discriminator_real_loss=1.069, discriminator_fake_loss=1.469, generator_loss=37.39, generator_mel_loss=28.87, generator_kl_loss=2.001, generator_dur_loss=1.658, generator_adv_loss=2.23, generator_feat_match_loss=2.631, over 126.00 samples.], tot_loss[discriminator_loss=2.642, discriminator_real_loss=1.359, discriminator_fake_loss=1.283, generator_loss=37.08, generator_mel_loss=28.91, generator_kl_loss=2.007, generator_dur_loss=1.694, generator_adv_loss=2.029, generator_feat_match_loss=2.434, over 1553.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:56:15,977 INFO [train.py:845] (2/4) Start epoch 38
2024-02-22 11:59:17,012 INFO [train.py:471] (2/4) Epoch 38, batch 31, global_batch_idx: 1400, batch size: 79, loss[discriminator_loss=2.592, discriminator_real_loss=1.064, discriminator_fake_loss=1.527, generator_loss=37.58, generator_mel_loss=28.91, generator_kl_loss=1.999, generator_dur_loss=1.721, generator_adv_loss=2.359, generator_feat_match_loss=2.594, over 79.00 samples.], tot_loss[discriminator_loss=2.631, discriminator_real_loss=1.374, discriminator_fake_loss=1.257, generator_loss=36.53, generator_mel_loss=28.4, generator_kl_loss=1.966, generator_dur_loss=1.71, generator_adv_loss=2.011, generator_feat_match_loss=2.445, over 2387.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 11:59:17,014 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 11:59:25,365 INFO [train.py:534] (2/4) Epoch 38, validation: discriminator_loss=2.544, discriminator_real_loss=1.505, discriminator_fake_loss=1.039, generator_loss=37.82, generator_mel_loss=29.34, generator_kl_loss=1.938, generator_dur_loss=1.725, generator_adv_loss=2.223, generator_feat_match_loss=2.589, over 100.00 samples.
2024-02-22 11:59:25,365 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28084MB
2024-02-22 11:59:49,919 INFO [train.py:845] (2/4) Start epoch 39
2024-02-22 12:03:20,058 INFO [train.py:845] (2/4) Start epoch 40
2024-02-22 12:04:18,764 INFO [train.py:471] (2/4) Epoch 40, batch 7, global_batch_idx: 1450, batch size: 52, loss[discriminator_loss=2.746, discriminator_real_loss=1.398, discriminator_fake_loss=1.347, generator_loss=36.99, generator_mel_loss=29.17, generator_kl_loss=1.986, generator_dur_loss=1.714, generator_adv_loss=1.708, generator_feat_match_loss=2.414, over 52.00 samples.], tot_loss[discriminator_loss=2.672, discriminator_real_loss=1.358, discriminator_fake_loss=1.313, generator_loss=35.97, generator_mel_loss=28.05, generator_kl_loss=1.95, generator_dur_loss=1.701, generator_adv_loss=1.991, generator_feat_match_loss=2.282, over 624.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:06:49,798 INFO [train.py:845] (2/4) Start epoch 41
2024-02-22 12:08:55,616 INFO [train.py:471] (2/4) Epoch 41, batch 20, global_batch_idx: 1500, batch size: 110, loss[discriminator_loss=2.59, discriminator_real_loss=1.459, discriminator_fake_loss=1.131, generator_loss=37.25, generator_mel_loss=29.17, generator_kl_loss=1.975, generator_dur_loss=1.711, generator_adv_loss=1.808, generator_feat_match_loss=2.582, over 110.00 samples.], tot_loss[discriminator_loss=2.668, discriminator_real_loss=1.385, discriminator_fake_loss=1.284, generator_loss=35.84, generator_mel_loss=27.9, generator_kl_loss=1.983, generator_dur_loss=1.715, generator_adv_loss=1.959, generator_feat_match_loss=2.285, over 1527.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:10:24,605 INFO [train.py:845] (2/4) Start epoch 42
2024-02-22 12:13:39,206 INFO [train.py:471] (2/4) Epoch 42, batch 33, global_batch_idx: 1550, batch size: 50, loss[discriminator_loss=2.703, discriminator_real_loss=1.333, discriminator_fake_loss=1.369, generator_loss=35.87, generator_mel_loss=28.02, generator_kl_loss=1.953, generator_dur_loss=1.711, generator_adv_loss=1.993, generator_feat_match_loss=2.191, over 50.00 samples.], tot_loss[discriminator_loss=2.712, discriminator_real_loss=1.421, discriminator_fake_loss=1.29, generator_loss=35.62, generator_mel_loss=27.77, generator_kl_loss=1.993, generator_dur_loss=1.712, generator_adv_loss=1.951, generator_feat_match_loss=2.198, over 2611.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:13:55,968 INFO [train.py:845] (2/4) Start epoch 43
2024-02-22 12:17:18,574 INFO [train.py:845] (2/4) Start epoch 44
2024-02-22 12:18:21,659 INFO [train.py:471] (2/4) Epoch 44, batch 9, global_batch_idx: 1600, batch size: 52, loss[discriminator_loss=2.895, discriminator_real_loss=1.652, discriminator_fake_loss=1.243, generator_loss=32.96, generator_mel_loss=25.82, generator_kl_loss=2.004, generator_dur_loss=1.759, generator_adv_loss=1.721, generator_feat_match_loss=1.652, over 52.00 samples.], tot_loss[discriminator_loss=2.757, discriminator_real_loss=1.435, discriminator_fake_loss=1.323, generator_loss=35.35, generator_mel_loss=27.66, generator_kl_loss=1.974, generator_dur_loss=1.719, generator_adv_loss=1.921, generator_feat_match_loss=2.078, over 724.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:18:21,661 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 12:18:30,082 INFO [train.py:534] (2/4) Epoch 44, validation: discriminator_loss=2.681, discriminator_real_loss=1.301, discriminator_fake_loss=1.381, generator_loss=36.55, generator_mel_loss=28.82, generator_kl_loss=2.043, generator_dur_loss=1.718, generator_adv_loss=1.761, generator_feat_match_loss=2.205, over 100.00 samples.
2024-02-22 12:18:30,083 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28084MB
2024-02-22 12:20:49,822 INFO [train.py:845] (2/4) Start epoch 45
2024-02-22 12:22:59,802 INFO [train.py:471] (2/4) Epoch 45, batch 22, global_batch_idx: 1650, batch size: 67, loss[discriminator_loss=2.656, discriminator_real_loss=1.419, discriminator_fake_loss=1.238, generator_loss=36.74, generator_mel_loss=28.44, generator_kl_loss=1.958, generator_dur_loss=1.732, generator_adv_loss=2.113, generator_feat_match_loss=2.498, over 67.00 samples.], tot_loss[discriminator_loss=2.696, discriminator_real_loss=1.422, discriminator_fake_loss=1.274, generator_loss=35.24, generator_mel_loss=27.37, generator_kl_loss=1.993, generator_dur_loss=1.712, generator_adv_loss=1.96, generator_feat_match_loss=2.208, over 1720.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:24:14,743 INFO [train.py:845] (2/4) Start epoch 46
2024-02-22 12:27:39,531 INFO [train.py:471] (2/4) Epoch 46, batch 35, global_batch_idx: 1700, batch size: 63, loss[discriminator_loss=2.656, discriminator_real_loss=1.344, discriminator_fake_loss=1.313, generator_loss=34.38, generator_mel_loss=26.56, generator_kl_loss=1.946, generator_dur_loss=1.724, generator_adv_loss=2.047, generator_feat_match_loss=2.105, over 63.00 samples.], tot_loss[discriminator_loss=2.733, discriminator_real_loss=1.399, discriminator_fake_loss=1.334, generator_loss=34.82, generator_mel_loss=27.12, generator_kl_loss=2.018, generator_dur_loss=1.709, generator_adv_loss=1.911, generator_feat_match_loss=2.07, over 2709.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:27:44,197 INFO [train.py:845] (2/4) Start epoch 47
2024-02-22 12:31:12,239 INFO [train.py:845] (2/4) Start epoch 48
2024-02-22 12:32:24,204 INFO [train.py:471] (2/4) Epoch 48, batch 11, global_batch_idx: 1750, batch size: 61, loss[discriminator_loss=2.668, discriminator_real_loss=1.348, discriminator_fake_loss=1.32, generator_loss=32.36, generator_mel_loss=24.98, generator_kl_loss=2.042, generator_dur_loss=1.736, generator_adv_loss=1.81, generator_feat_match_loss=1.788, over 61.00 samples.], tot_loss[discriminator_loss=2.728, discriminator_real_loss=1.396, discriminator_fake_loss=1.331, generator_loss=34.58, generator_mel_loss=26.78, generator_kl_loss=2.028, generator_dur_loss=1.712, generator_adv_loss=1.925, generator_feat_match_loss=2.134, over 907.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:34:37,225 INFO [train.py:845] (2/4) Start epoch 49
2024-02-22 12:37:02,006 INFO [train.py:471] (2/4) Epoch 49, batch 24, global_batch_idx: 1800, batch size: 154, loss[discriminator_loss=2.77, discriminator_real_loss=1.387, discriminator_fake_loss=1.382, generator_loss=35.72, generator_mel_loss=27.74, generator_kl_loss=2.034, generator_dur_loss=1.658, generator_adv_loss=2.027, generator_feat_match_loss=2.254, over 154.00 samples.], tot_loss[discriminator_loss=2.744, discriminator_real_loss=1.439, discriminator_fake_loss=1.304, generator_loss=34.41, generator_mel_loss=26.68, generator_kl_loss=2.041, generator_dur_loss=1.712, generator_adv_loss=1.923, generator_feat_match_loss=2.057, over 1879.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:37:02,008 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 12:37:10,140 INFO [train.py:534] (2/4) Epoch 49, validation: discriminator_loss=2.597, discriminator_real_loss=1.504, discriminator_fake_loss=1.093, generator_loss=35.51, generator_mel_loss=27.49, generator_kl_loss=2.174, generator_dur_loss=1.713, generator_adv_loss=2.055, generator_feat_match_loss=2.071, over 100.00 samples.
2024-02-22 12:37:10,141 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 12:38:16,327 INFO [train.py:845] (2/4) Start epoch 50
2024-02-22 12:41:45,063 INFO [train.py:845] (2/4) Start epoch 51
2024-02-22 12:41:58,562 INFO [train.py:471] (2/4) Epoch 51, batch 0, global_batch_idx: 1850, batch size: 79, loss[discriminator_loss=2.918, discriminator_real_loss=1.118, discriminator_fake_loss=1.801, generator_loss=34.27, generator_mel_loss=26.25, generator_kl_loss=2.006, generator_dur_loss=1.718, generator_adv_loss=2.182, generator_feat_match_loss=2.115, over 79.00 samples.], tot_loss[discriminator_loss=2.918, discriminator_real_loss=1.118, discriminator_fake_loss=1.801, generator_loss=34.27, generator_mel_loss=26.25, generator_kl_loss=2.006, generator_dur_loss=1.718, generator_adv_loss=2.182, generator_feat_match_loss=2.115, over 79.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:45:13,349 INFO [train.py:845] (2/4) Start epoch 52
2024-02-22 12:46:33,547 INFO [train.py:471] (2/4) Epoch 52, batch 13, global_batch_idx: 1900, batch size: 65, loss[discriminator_loss=2.73, discriminator_real_loss=1.316, discriminator_fake_loss=1.414, generator_loss=33.83, generator_mel_loss=25.99, generator_kl_loss=1.953, generator_dur_loss=1.735, generator_adv_loss=2.09, generator_feat_match_loss=2.061, over 65.00 samples.], tot_loss[discriminator_loss=2.713, discriminator_real_loss=1.403, discriminator_fake_loss=1.31, generator_loss=33.87, generator_mel_loss=26.13, generator_kl_loss=2.038, generator_dur_loss=1.728, generator_adv_loss=1.896, generator_feat_match_loss=2.076, over 914.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:48:38,836 INFO [train.py:845] (2/4) Start epoch 53
2024-02-22 12:51:14,966 INFO [train.py:471] (2/4) Epoch 53, batch 26, global_batch_idx: 1950, batch size: 52, loss[discriminator_loss=2.812, discriminator_real_loss=1.7, discriminator_fake_loss=1.112, generator_loss=33.27, generator_mel_loss=25.64, generator_kl_loss=2.107, generator_dur_loss=1.736, generator_adv_loss=1.783, generator_feat_match_loss=2.002, over 52.00 samples.], tot_loss[discriminator_loss=2.724, discriminator_real_loss=1.406, discriminator_fake_loss=1.318, generator_loss=33.9, generator_mel_loss=26.12, generator_kl_loss=2.054, generator_dur_loss=1.716, generator_adv_loss=1.909, generator_feat_match_loss=2.098, over 1925.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 32.0
2024-02-22 12:52:07,328 INFO [train.py:845] (2/4) Start epoch 54
2024-02-22 12:55:30,273 INFO [train.py:845] (2/4) Start epoch 55
2024-02-22 12:55:52,232 INFO [train.py:471] (2/4) Epoch 55, batch 2, global_batch_idx: 2000, batch size: 65, loss[discriminator_loss=2.785, discriminator_real_loss=1.504, discriminator_fake_loss=1.281, generator_loss=33.37, generator_mel_loss=25.77, generator_kl_loss=2.051, generator_dur_loss=1.737, generator_adv_loss=1.728, generator_feat_match_loss=2.08, over 65.00 samples.], tot_loss[discriminator_loss=2.799, discriminator_real_loss=1.458, discriminator_fake_loss=1.341, generator_loss=33.9, generator_mel_loss=26.16, generator_kl_loss=2.043, generator_dur_loss=1.741, generator_adv_loss=1.868, generator_feat_match_loss=2.095, over 180.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 64.0
2024-02-22 12:55:52,232 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 12:56:01,104 INFO [train.py:534] (2/4) Epoch 55, validation: discriminator_loss=2.742, discriminator_real_loss=1.243, discriminator_fake_loss=1.499, generator_loss=34.87, generator_mel_loss=27.47, generator_kl_loss=1.89, generator_dur_loss=1.71, generator_adv_loss=1.667, generator_feat_match_loss=2.139, over 100.00 samples.
2024-02-22 12:56:01,105 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 12:59:05,880 INFO [train.py:845] (2/4) Start epoch 56
2024-02-22 13:00:42,759 INFO [train.py:471] (2/4) Epoch 56, batch 15, global_batch_idx: 2050, batch size: 76, loss[discriminator_loss=2.727, discriminator_real_loss=1.246, discriminator_fake_loss=1.48, generator_loss=34.58, generator_mel_loss=26.68, generator_kl_loss=2.044, generator_dur_loss=1.725, generator_adv_loss=1.821, generator_feat_match_loss=2.309, over 76.00 samples.], tot_loss[discriminator_loss=2.71, discriminator_real_loss=1.412, discriminator_fake_loss=1.298, generator_loss=33.97, generator_mel_loss=26.14, generator_kl_loss=2.056, generator_dur_loss=1.707, generator_adv_loss=1.886, generator_feat_match_loss=2.179, over 1268.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 64.0
2024-02-22 13:02:34,991 INFO [train.py:845] (2/4) Start epoch 57
2024-02-22 13:05:29,683 INFO [train.py:471] (2/4) Epoch 57, batch 28, global_batch_idx: 2100, batch size: 51, loss[discriminator_loss=2.73, discriminator_real_loss=1.191, discriminator_fake_loss=1.538, generator_loss=33.76, generator_mel_loss=25.68, generator_kl_loss=2.033, generator_dur_loss=1.732, generator_adv_loss=2.004, generator_feat_match_loss=2.312, over 51.00 samples.], tot_loss[discriminator_loss=2.726, discriminator_real_loss=1.391, discriminator_fake_loss=1.334, generator_loss=33.88, generator_mel_loss=25.97, generator_kl_loss=2.072, generator_dur_loss=1.703, generator_adv_loss=1.949, generator_feat_match_loss=2.188, over 2390.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 64.0
2024-02-22 13:06:08,352 INFO [train.py:845] (2/4) Start epoch 58
2024-02-22 13:09:41,030 INFO [train.py:845] (2/4) Start epoch 59
2024-02-22 13:10:14,127 INFO [train.py:471] (2/4) Epoch 59, batch 4, global_batch_idx: 2150, batch size: 67, loss[discriminator_loss=2.789, discriminator_real_loss=1.787, discriminator_fake_loss=1.001, generator_loss=33.65, generator_mel_loss=26.04, generator_kl_loss=2.131, generator_dur_loss=1.737, generator_adv_loss=1.56, generator_feat_match_loss=2.182, over 67.00 samples.], tot_loss[discriminator_loss=2.727, discriminator_real_loss=1.514, discriminator_fake_loss=1.213, generator_loss=33.97, generator_mel_loss=25.93, generator_kl_loss=2.058, generator_dur_loss=1.71, generator_adv_loss=2.018, generator_feat_match_loss=2.257, over 428.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 64.0
2024-02-22 13:13:11,904 INFO [train.py:845] (2/4) Start epoch 60
2024-02-22 13:15:04,492 INFO [train.py:471] (2/4) Epoch 60, batch 17, global_batch_idx: 2200, batch size: 73, loss[discriminator_loss=2.746, discriminator_real_loss=1.416, discriminator_fake_loss=1.33, generator_loss=33.42, generator_mel_loss=25.86, generator_kl_loss=2.106, generator_dur_loss=1.693, generator_adv_loss=1.815, generator_feat_match_loss=1.95, over 73.00 samples.], tot_loss[discriminator_loss=2.756, discriminator_real_loss=1.407, discriminator_fake_loss=1.349, generator_loss=33.77, generator_mel_loss=25.95, generator_kl_loss=2.064, generator_dur_loss=1.695, generator_adv_loss=1.935, generator_feat_match_loss=2.125, over 1402.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 64.0
2024-02-22 13:15:04,493 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 13:15:13,046 INFO [train.py:534] (2/4) Epoch 60, validation: discriminator_loss=2.683, discriminator_real_loss=1.277, discriminator_fake_loss=1.406, generator_loss=34.43, generator_mel_loss=26.87, generator_kl_loss=2.029, generator_dur_loss=1.702, generator_adv_loss=1.714, generator_feat_match_loss=2.111, over 100.00 samples.
2024-02-22 13:15:13,047 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 13:16:51,453 INFO [train.py:845] (2/4) Start epoch 61
2024-02-22 13:19:57,285 INFO [train.py:471] (2/4) Epoch 61, batch 30, global_batch_idx: 2250, batch size: 53, loss[discriminator_loss=2.812, discriminator_real_loss=1.488, discriminator_fake_loss=1.324, generator_loss=31.62, generator_mel_loss=24.33, generator_kl_loss=2.022, generator_dur_loss=1.701, generator_adv_loss=1.797, generator_feat_match_loss=1.766, over 53.00 samples.], tot_loss[discriminator_loss=2.729, discriminator_real_loss=1.393, discriminator_fake_loss=1.336, generator_loss=33.28, generator_mel_loss=25.54, generator_kl_loss=2.013, generator_dur_loss=1.692, generator_adv_loss=1.906, generator_feat_match_loss=2.134, over 2255.00 samples.], cur_lr_g: 1.99e-04, cur_lr_d: 1.99e-04, grad_scale: 64.0
2024-02-22 13:20:23,975 INFO [train.py:845] (2/4) Start epoch 62
2024-02-22 13:23:58,726 INFO [train.py:845] (2/4) Start epoch 63
2024-02-22 13:24:49,434 INFO [train.py:471] (2/4) Epoch 63, batch 6, global_batch_idx: 2300, batch size: 153, loss[discriminator_loss=2.695, discriminator_real_loss=1.32, discriminator_fake_loss=1.374, generator_loss=33.54, generator_mel_loss=25.6, generator_kl_loss=1.962, generator_dur_loss=1.66, generator_adv_loss=2.133, generator_feat_match_loss=2.189, over 153.00 samples.], tot_loss[discriminator_loss=2.684, discriminator_real_loss=1.378, discriminator_fake_loss=1.305, generator_loss=32.99, generator_mel_loss=25.26, generator_kl_loss=1.965, generator_dur_loss=1.659, generator_adv_loss=1.913, generator_feat_match_loss=2.192, over 650.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:27:31,890 INFO [train.py:845] (2/4) Start epoch 64
2024-02-22 13:29:32,881 INFO [train.py:471] (2/4) Epoch 64, batch 19, global_batch_idx: 2350, batch size: 95, loss[discriminator_loss=2.727, discriminator_real_loss=1.526, discriminator_fake_loss=1.199, generator_loss=33.2, generator_mel_loss=25.46, generator_kl_loss=1.881, generator_dur_loss=1.65, generator_adv_loss=2.078, generator_feat_match_loss=2.129, over 95.00 samples.], tot_loss[discriminator_loss=2.729, discriminator_real_loss=1.406, discriminator_fake_loss=1.324, generator_loss=32.78, generator_mel_loss=25.16, generator_kl_loss=1.943, generator_dur_loss=1.665, generator_adv_loss=1.885, generator_feat_match_loss=2.126, over 1586.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:31:00,555 INFO [train.py:845] (2/4) Start epoch 65
2024-02-22 13:34:12,392 INFO [train.py:471] (2/4) Epoch 65, batch 32, global_batch_idx: 2400, batch size: 101, loss[discriminator_loss=2.65, discriminator_real_loss=1.461, discriminator_fake_loss=1.189, generator_loss=32.93, generator_mel_loss=25.25, generator_kl_loss=2.069, generator_dur_loss=1.638, generator_adv_loss=1.905, generator_feat_match_loss=2.064, over 101.00 samples.], tot_loss[discriminator_loss=2.761, discriminator_real_loss=1.416, discriminator_fake_loss=1.346, generator_loss=32.64, generator_mel_loss=25.05, generator_kl_loss=1.93, generator_dur_loss=1.655, generator_adv_loss=1.903, generator_feat_match_loss=2.1, over 2586.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:34:12,394 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 13:34:21,034 INFO [train.py:534] (2/4) Epoch 65, validation: discriminator_loss=2.655, discriminator_real_loss=1.323, discriminator_fake_loss=1.333, generator_loss=33.61, generator_mel_loss=26.13, generator_kl_loss=1.947, generator_dur_loss=1.639, generator_adv_loss=1.815, generator_feat_match_loss=2.078, over 100.00 samples.
2024-02-22 13:34:21,035 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 13:34:38,724 INFO [train.py:845] (2/4) Start epoch 66
2024-02-22 13:38:07,979 INFO [train.py:845] (2/4) Start epoch 67
2024-02-22 13:39:01,375 INFO [train.py:471] (2/4) Epoch 67, batch 8, global_batch_idx: 2450, batch size: 58, loss[discriminator_loss=2.797, discriminator_real_loss=1.343, discriminator_fake_loss=1.453, generator_loss=32.12, generator_mel_loss=24.66, generator_kl_loss=1.856, generator_dur_loss=1.641, generator_adv_loss=1.825, generator_feat_match_loss=2.137, over 58.00 samples.], tot_loss[discriminator_loss=2.717, discriminator_real_loss=1.357, discriminator_fake_loss=1.36, generator_loss=32.39, generator_mel_loss=24.72, generator_kl_loss=1.94, generator_dur_loss=1.654, generator_adv_loss=1.927, generator_feat_match_loss=2.145, over 554.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:41:41,720 INFO [train.py:845] (2/4) Start epoch 68
2024-02-22 13:43:55,912 INFO [train.py:471] (2/4) Epoch 68, batch 21, global_batch_idx: 2500, batch size: 61, loss[discriminator_loss=2.684, discriminator_real_loss=1.346, discriminator_fake_loss=1.338, generator_loss=32.89, generator_mel_loss=25.1, generator_kl_loss=2.051, generator_dur_loss=1.639, generator_adv_loss=1.879, generator_feat_match_loss=2.219, over 61.00 samples.], tot_loss[discriminator_loss=2.785, discriminator_real_loss=1.417, discriminator_fake_loss=1.368, generator_loss=32.41, generator_mel_loss=24.9, generator_kl_loss=1.968, generator_dur_loss=1.631, generator_adv_loss=1.872, generator_feat_match_loss=2.039, over 1818.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:45:14,353 INFO [train.py:845] (2/4) Start epoch 69
2024-02-22 13:48:33,771 INFO [train.py:471] (2/4) Epoch 69, batch 34, global_batch_idx: 2550, batch size: 69, loss[discriminator_loss=2.75, discriminator_real_loss=1.557, discriminator_fake_loss=1.193, generator_loss=31.46, generator_mel_loss=23.97, generator_kl_loss=1.968, generator_dur_loss=1.628, generator_adv_loss=1.784, generator_feat_match_loss=2.104, over 69.00 samples.], tot_loss[discriminator_loss=2.764, discriminator_real_loss=1.419, discriminator_fake_loss=1.344, generator_loss=32.19, generator_mel_loss=24.7, generator_kl_loss=1.935, generator_dur_loss=1.633, generator_adv_loss=1.879, generator_feat_match_loss=2.039, over 2407.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:48:44,257 INFO [train.py:845] (2/4) Start epoch 70
2024-02-22 13:52:04,534 INFO [train.py:845] (2/4) Start epoch 71
2024-02-22 13:53:11,309 INFO [train.py:471] (2/4) Epoch 71, batch 10, global_batch_idx: 2600, batch size: 65, loss[discriminator_loss=2.639, discriminator_real_loss=1.556, discriminator_fake_loss=1.083, generator_loss=33.18, generator_mel_loss=25.55, generator_kl_loss=2.001, generator_dur_loss=1.633, generator_adv_loss=1.72, generator_feat_match_loss=2.279, over 65.00 samples.], tot_loss[discriminator_loss=2.721, discriminator_real_loss=1.391, discriminator_fake_loss=1.33, generator_loss=32.49, generator_mel_loss=24.81, generator_kl_loss=1.949, generator_dur_loss=1.631, generator_adv_loss=1.917, generator_feat_match_loss=2.174, over 772.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:53:11,310 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 13:53:19,566 INFO [train.py:534] (2/4) Epoch 71, validation: discriminator_loss=2.614, discriminator_real_loss=1.109, discriminator_fake_loss=1.505, generator_loss=33.16, generator_mel_loss=25.5, generator_kl_loss=1.929, generator_dur_loss=1.613, generator_adv_loss=1.652, generator_feat_match_loss=2.469, over 100.00 samples.
2024-02-22 13:53:19,567 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 13:55:36,865 INFO [train.py:845] (2/4) Start epoch 72
2024-02-22 13:58:01,579 INFO [train.py:471] (2/4) Epoch 72, batch 23, global_batch_idx: 2650, batch size: 126, loss[discriminator_loss=2.676, discriminator_real_loss=1.334, discriminator_fake_loss=1.343, generator_loss=32.7, generator_mel_loss=25.16, generator_kl_loss=1.939, generator_dur_loss=1.599, generator_adv_loss=1.717, generator_feat_match_loss=2.285, over 126.00 samples.], tot_loss[discriminator_loss=2.749, discriminator_real_loss=1.408, discriminator_fake_loss=1.341, generator_loss=32.29, generator_mel_loss=24.73, generator_kl_loss=1.937, generator_dur_loss=1.611, generator_adv_loss=1.872, generator_feat_match_loss=2.14, over 2001.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 13:59:05,102 INFO [train.py:845] (2/4) Start epoch 73
2024-02-22 14:02:35,912 INFO [train.py:471] (2/4) Epoch 73, batch 36, global_batch_idx: 2700, batch size: 90, loss[discriminator_loss=2.758, discriminator_real_loss=1.167, discriminator_fake_loss=1.59, generator_loss=32.46, generator_mel_loss=24.72, generator_kl_loss=1.922, generator_dur_loss=1.622, generator_adv_loss=1.943, generator_feat_match_loss=2.248, over 90.00 samples.], tot_loss[discriminator_loss=2.71, discriminator_real_loss=1.374, discriminator_fake_loss=1.335, generator_loss=32.25, generator_mel_loss=24.59, generator_kl_loss=1.956, generator_dur_loss=1.61, generator_adv_loss=1.887, generator_feat_match_loss=2.208, over 2889.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:02:36,412 INFO [train.py:845] (2/4) Start epoch 74
2024-02-22 14:06:09,444 INFO [train.py:845] (2/4) Start epoch 75
2024-02-22 14:07:25,725 INFO [train.py:471] (2/4) Epoch 75, batch 12, global_batch_idx: 2750, batch size: 90, loss[discriminator_loss=2.658, discriminator_real_loss=1.27, discriminator_fake_loss=1.389, generator_loss=32.65, generator_mel_loss=24.8, generator_kl_loss=1.975, generator_dur_loss=1.594, generator_adv_loss=2.025, generator_feat_match_loss=2.256, over 90.00 samples.], tot_loss[discriminator_loss=2.714, discriminator_real_loss=1.394, discriminator_fake_loss=1.321, generator_loss=32.01, generator_mel_loss=24.35, generator_kl_loss=1.905, generator_dur_loss=1.608, generator_adv_loss=1.92, generator_feat_match_loss=2.228, over 897.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:09:32,536 INFO [train.py:845] (2/4) Start epoch 76
2024-02-22 14:12:01,225 INFO [train.py:471] (2/4) Epoch 76, batch 25, global_batch_idx: 2800, batch size: 59, loss[discriminator_loss=2.734, discriminator_real_loss=1.527, discriminator_fake_loss=1.207, generator_loss=31.63, generator_mel_loss=24.26, generator_kl_loss=1.844, generator_dur_loss=1.608, generator_adv_loss=1.754, generator_feat_match_loss=2.16, over 59.00 samples.], tot_loss[discriminator_loss=2.749, discriminator_real_loss=1.409, discriminator_fake_loss=1.34, generator_loss=31.78, generator_mel_loss=24.29, generator_kl_loss=1.938, generator_dur_loss=1.611, generator_adv_loss=1.873, generator_feat_match_loss=2.077, over 1897.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:12:01,226 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 14:12:09,538 INFO [train.py:534] (2/4) Epoch 76, validation: discriminator_loss=2.756, discriminator_real_loss=1.335, discriminator_fake_loss=1.421, generator_loss=33.14, generator_mel_loss=25.57, generator_kl_loss=1.976, generator_dur_loss=1.591, generator_adv_loss=1.757, generator_feat_match_loss=2.245, over 100.00 samples.
2024-02-22 14:12:09,539 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 14:13:09,423 INFO [train.py:845] (2/4) Start epoch 77
2024-02-22 14:16:35,673 INFO [train.py:845] (2/4) Start epoch 78
2024-02-22 14:16:54,882 INFO [train.py:471] (2/4) Epoch 78, batch 1, global_batch_idx: 2850, batch size: 52, loss[discriminator_loss=2.758, discriminator_real_loss=1.394, discriminator_fake_loss=1.363, generator_loss=32.5, generator_mel_loss=24.72, generator_kl_loss=1.904, generator_dur_loss=1.619, generator_adv_loss=2.141, generator_feat_match_loss=2.111, over 52.00 samples.], tot_loss[discriminator_loss=2.722, discriminator_real_loss=1.459, discriminator_fake_loss=1.263, generator_loss=31.99, generator_mel_loss=24.36, generator_kl_loss=1.904, generator_dur_loss=1.62, generator_adv_loss=1.984, generator_feat_match_loss=2.124, over 125.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:20:02,359 INFO [train.py:845] (2/4) Start epoch 79
2024-02-22 14:21:28,331 INFO [train.py:471] (2/4) Epoch 79, batch 14, global_batch_idx: 2900, batch size: 63, loss[discriminator_loss=2.785, discriminator_real_loss=1.48, discriminator_fake_loss=1.305, generator_loss=31.94, generator_mel_loss=24.31, generator_kl_loss=1.941, generator_dur_loss=1.612, generator_adv_loss=1.72, generator_feat_match_loss=2.352, over 63.00 samples.], tot_loss[discriminator_loss=2.732, discriminator_real_loss=1.398, discriminator_fake_loss=1.334, generator_loss=32.08, generator_mel_loss=24.37, generator_kl_loss=1.973, generator_dur_loss=1.602, generator_adv_loss=1.923, generator_feat_match_loss=2.216, over 1179.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:23:31,856 INFO [train.py:845] (2/4) Start epoch 80
2024-02-22 14:26:12,972 INFO [train.py:471] (2/4) Epoch 80, batch 27, global_batch_idx: 2950, batch size: 81, loss[discriminator_loss=2.73, discriminator_real_loss=1.434, discriminator_fake_loss=1.296, generator_loss=31.99, generator_mel_loss=24.31, generator_kl_loss=2.028, generator_dur_loss=1.594, generator_adv_loss=1.755, generator_feat_match_loss=2.303, over 81.00 samples.], tot_loss[discriminator_loss=2.72, discriminator_real_loss=1.39, discriminator_fake_loss=1.33, generator_loss=31.97, generator_mel_loss=24.26, generator_kl_loss=1.949, generator_dur_loss=1.594, generator_adv_loss=1.912, generator_feat_match_loss=2.26, over 2199.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:26:56,135 INFO [train.py:845] (2/4) Start epoch 81
2024-02-22 14:30:29,731 INFO [train.py:845] (2/4) Start epoch 82
2024-02-22 14:30:58,404 INFO [train.py:471] (2/4) Epoch 82, batch 3, global_batch_idx: 3000, batch size: 53, loss[discriminator_loss=2.723, discriminator_real_loss=1.345, discriminator_fake_loss=1.377, generator_loss=31.93, generator_mel_loss=24.17, generator_kl_loss=1.961, generator_dur_loss=1.618, generator_adv_loss=1.945, generator_feat_match_loss=2.24, over 53.00 samples.], tot_loss[discriminator_loss=2.714, discriminator_real_loss=1.413, discriminator_fake_loss=1.301, generator_loss=31.78, generator_mel_loss=24.07, generator_kl_loss=1.952, generator_dur_loss=1.605, generator_adv_loss=1.895, generator_feat_match_loss=2.26, over 277.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:30:58,406 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 14:31:07,207 INFO [train.py:534] (2/4) Epoch 82, validation: discriminator_loss=2.727, discriminator_real_loss=1.361, discriminator_fake_loss=1.366, generator_loss=32.9, generator_mel_loss=25.12, generator_kl_loss=1.994, generator_dur_loss=1.576, generator_adv_loss=1.835, generator_feat_match_loss=2.378, over 100.00 samples.
2024-02-22 14:31:07,208 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 14:34:06,852 INFO [train.py:845] (2/4) Start epoch 83
2024-02-22 14:35:51,164 INFO [train.py:471] (2/4) Epoch 83, batch 16, global_batch_idx: 3050, batch size: 126, loss[discriminator_loss=2.691, discriminator_real_loss=1.245, discriminator_fake_loss=1.445, generator_loss=32.38, generator_mel_loss=24.52, generator_kl_loss=2.025, generator_dur_loss=1.57, generator_adv_loss=2.006, generator_feat_match_loss=2.264, over 126.00 samples.], tot_loss[discriminator_loss=2.74, discriminator_real_loss=1.398, discriminator_fake_loss=1.342, generator_loss=31.72, generator_mel_loss=24.07, generator_kl_loss=1.948, generator_dur_loss=1.591, generator_adv_loss=1.903, generator_feat_match_loss=2.212, over 1204.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:37:41,534 INFO [train.py:845] (2/4) Start epoch 84
2024-02-22 14:40:26,956 INFO [train.py:471] (2/4) Epoch 84, batch 29, global_batch_idx: 3100, batch size: 54, loss[discriminator_loss=2.732, discriminator_real_loss=1.436, discriminator_fake_loss=1.297, generator_loss=31.61, generator_mel_loss=23.9, generator_kl_loss=1.961, generator_dur_loss=1.622, generator_adv_loss=1.813, generator_feat_match_loss=2.316, over 54.00 samples.], tot_loss[discriminator_loss=2.712, discriminator_real_loss=1.384, discriminator_fake_loss=1.328, generator_loss=31.62, generator_mel_loss=23.97, generator_kl_loss=1.932, generator_dur_loss=1.596, generator_adv_loss=1.899, generator_feat_match_loss=2.227, over 2099.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:41:09,844 INFO [train.py:845] (2/4) Start epoch 85
2024-02-22 14:44:35,832 INFO [train.py:845] (2/4) Start epoch 86
2024-02-22 14:45:12,215 INFO [train.py:471] (2/4) Epoch 86, batch 5, global_batch_idx: 3150, batch size: 61, loss[discriminator_loss=2.732, discriminator_real_loss=1.531, discriminator_fake_loss=1.201, generator_loss=30.83, generator_mel_loss=23.64, generator_kl_loss=1.936, generator_dur_loss=1.587, generator_adv_loss=1.736, generator_feat_match_loss=1.939, over 61.00 samples.], tot_loss[discriminator_loss=2.748, discriminator_real_loss=1.444, discriminator_fake_loss=1.303, generator_loss=31.23, generator_mel_loss=23.87, generator_kl_loss=1.956, generator_dur_loss=1.601, generator_adv_loss=1.856, generator_feat_match_loss=1.952, over 363.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:48:04,872 INFO [train.py:845] (2/4) Start epoch 87
2024-02-22 14:49:58,504 INFO [train.py:471] (2/4) Epoch 87, batch 18, global_batch_idx: 3200, batch size: 58, loss[discriminator_loss=2.723, discriminator_real_loss=1.504, discriminator_fake_loss=1.219, generator_loss=31.72, generator_mel_loss=24.03, generator_kl_loss=1.889, generator_dur_loss=1.591, generator_adv_loss=1.812, generator_feat_match_loss=2.391, over 58.00 samples.], tot_loss[discriminator_loss=2.697, discriminator_real_loss=1.394, discriminator_fake_loss=1.304, generator_loss=31.65, generator_mel_loss=24.02, generator_kl_loss=1.947, generator_dur_loss=1.588, generator_adv_loss=1.873, generator_feat_match_loss=2.229, over 1433.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:49:58,505 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 14:50:06,499 INFO [train.py:534] (2/4) Epoch 87, validation: discriminator_loss=2.679, discriminator_real_loss=1.362, discriminator_fake_loss=1.317, generator_loss=32.39, generator_mel_loss=24.82, generator_kl_loss=2.001, generator_dur_loss=1.584, generator_adv_loss=1.783, generator_feat_match_loss=2.204, over 100.00 samples.
2024-02-22 14:50:06,500 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 14:51:44,597 INFO [train.py:845] (2/4) Start epoch 88
2024-02-22 14:54:41,935 INFO [train.py:471] (2/4) Epoch 88, batch 31, global_batch_idx: 3250, batch size: 53, loss[discriminator_loss=2.771, discriminator_real_loss=1.432, discriminator_fake_loss=1.34, generator_loss=31.36, generator_mel_loss=23.54, generator_kl_loss=1.912, generator_dur_loss=1.578, generator_adv_loss=2.299, generator_feat_match_loss=2.025, over 53.00 samples.], tot_loss[discriminator_loss=2.721, discriminator_real_loss=1.388, discriminator_fake_loss=1.333, generator_loss=31.69, generator_mel_loss=23.94, generator_kl_loss=1.954, generator_dur_loss=1.58, generator_adv_loss=1.934, generator_feat_match_loss=2.283, over 2439.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 14:55:12,514 INFO [train.py:845] (2/4) Start epoch 89
2024-02-22 14:58:44,790 INFO [train.py:845] (2/4) Start epoch 90
2024-02-22 14:59:38,116 INFO [train.py:471] (2/4) Epoch 90, batch 7, global_batch_idx: 3300, batch size: 51, loss[discriminator_loss=2.758, discriminator_real_loss=1.414, discriminator_fake_loss=1.344, generator_loss=31.61, generator_mel_loss=24.22, generator_kl_loss=1.836, generator_dur_loss=1.601, generator_adv_loss=1.809, generator_feat_match_loss=2.137, over 51.00 samples.], tot_loss[discriminator_loss=2.713, discriminator_real_loss=1.403, discriminator_fake_loss=1.31, generator_loss=31.59, generator_mel_loss=23.84, generator_kl_loss=1.975, generator_dur_loss=1.589, generator_adv_loss=1.921, generator_feat_match_loss=2.265, over 561.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:02:19,748 INFO [train.py:845] (2/4) Start epoch 91
2024-02-22 15:04:21,587 INFO [train.py:471] (2/4) Epoch 91, batch 20, global_batch_idx: 3350, batch size: 53, loss[discriminator_loss=2.693, discriminator_real_loss=1.365, discriminator_fake_loss=1.328, generator_loss=31.2, generator_mel_loss=23.77, generator_kl_loss=1.862, generator_dur_loss=1.6, generator_adv_loss=1.829, generator_feat_match_loss=2.137, over 53.00 samples.], tot_loss[discriminator_loss=2.746, discriminator_real_loss=1.393, discriminator_fake_loss=1.353, generator_loss=31.78, generator_mel_loss=24.07, generator_kl_loss=1.94, generator_dur_loss=1.583, generator_adv_loss=1.91, generator_feat_match_loss=2.271, over 1582.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:05:44,006 INFO [train.py:845] (2/4) Start epoch 92
2024-02-22 15:08:48,643 INFO [train.py:471] (2/4) Epoch 92, batch 33, global_batch_idx: 3400, batch size: 126, loss[discriminator_loss=2.719, discriminator_real_loss=1.388, discriminator_fake_loss=1.331, generator_loss=31.73, generator_mel_loss=24.03, generator_kl_loss=1.89, generator_dur_loss=1.572, generator_adv_loss=1.861, generator_feat_match_loss=2.373, over 126.00 samples.], tot_loss[discriminator_loss=2.705, discriminator_real_loss=1.375, discriminator_fake_loss=1.33, generator_loss=31.44, generator_mel_loss=23.75, generator_kl_loss=1.955, generator_dur_loss=1.583, generator_adv_loss=1.884, generator_feat_match_loss=2.262, over 2438.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:08:48,645 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 15:08:57,063 INFO [train.py:534] (2/4) Epoch 92, validation: discriminator_loss=2.641, discriminator_real_loss=1.283, discriminator_fake_loss=1.359, generator_loss=32.14, generator_mel_loss=24.39, generator_kl_loss=1.946, generator_dur_loss=1.579, generator_adv_loss=1.815, generator_feat_match_loss=2.407, over 100.00 samples.
2024-02-22 15:08:57,064 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 15:09:15,002 INFO [train.py:845] (2/4) Start epoch 93
2024-02-22 15:12:46,213 INFO [train.py:845] (2/4) Start epoch 94
2024-02-22 15:13:46,465 INFO [train.py:471] (2/4) Epoch 94, batch 9, global_batch_idx: 3450, batch size: 58, loss[discriminator_loss=2.719, discriminator_real_loss=1.342, discriminator_fake_loss=1.377, generator_loss=31.99, generator_mel_loss=24.28, generator_kl_loss=1.909, generator_dur_loss=1.581, generator_adv_loss=1.896, generator_feat_match_loss=2.318, over 58.00 samples.], tot_loss[discriminator_loss=2.704, discriminator_real_loss=1.37, discriminator_fake_loss=1.334, generator_loss=31.44, generator_mel_loss=23.81, generator_kl_loss=1.911, generator_dur_loss=1.579, generator_adv_loss=1.882, generator_feat_match_loss=2.252, over 779.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:16:14,799 INFO [train.py:845] (2/4) Start epoch 95
2024-02-22 15:18:36,058 INFO [train.py:471] (2/4) Epoch 95, batch 22, global_batch_idx: 3500, batch size: 126, loss[discriminator_loss=2.801, discriminator_real_loss=1.24, discriminator_fake_loss=1.56, generator_loss=31.54, generator_mel_loss=24.04, generator_kl_loss=1.945, generator_dur_loss=1.554, generator_adv_loss=1.886, generator_feat_match_loss=2.119, over 126.00 samples.], tot_loss[discriminator_loss=2.751, discriminator_real_loss=1.387, discriminator_fake_loss=1.364, generator_loss=31.25, generator_mel_loss=23.67, generator_kl_loss=1.95, generator_dur_loss=1.576, generator_adv_loss=1.889, generator_feat_match_loss=2.158, over 1809.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:19:47,909 INFO [train.py:845] (2/4) Start epoch 96
2024-02-22 15:23:02,028 INFO [train.py:471] (2/4) Epoch 96, batch 35, global_batch_idx: 3550, batch size: 60, loss[discriminator_loss=2.754, discriminator_real_loss=1.557, discriminator_fake_loss=1.196, generator_loss=31.14, generator_mel_loss=23.9, generator_kl_loss=1.861, generator_dur_loss=1.605, generator_adv_loss=1.581, generator_feat_match_loss=2.188, over 60.00 samples.], tot_loss[discriminator_loss=2.757, discriminator_real_loss=1.424, discriminator_fake_loss=1.333, generator_loss=31.17, generator_mel_loss=23.55, generator_kl_loss=1.918, generator_dur_loss=1.581, generator_adv_loss=1.924, generator_feat_match_loss=2.199, over 2561.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:23:08,354 INFO [train.py:845] (2/4) Start epoch 97
2024-02-22 15:26:36,300 INFO [train.py:845] (2/4) Start epoch 98
2024-02-22 15:27:52,677 INFO [train.py:471] (2/4) Epoch 98, batch 11, global_batch_idx: 3600, batch size: 53, loss[discriminator_loss=2.68, discriminator_real_loss=1.243, discriminator_fake_loss=1.436, generator_loss=31.12, generator_mel_loss=23.31, generator_kl_loss=1.906, generator_dur_loss=1.606, generator_adv_loss=2.074, generator_feat_match_loss=2.219, over 53.00 samples.], tot_loss[discriminator_loss=2.771, discriminator_real_loss=1.435, discriminator_fake_loss=1.336, generator_loss=31.18, generator_mel_loss=23.48, generator_kl_loss=1.956, generator_dur_loss=1.569, generator_adv_loss=1.917, generator_feat_match_loss=2.255, over 955.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:27:52,678 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 15:28:01,166 INFO [train.py:534] (2/4) Epoch 98, validation: discriminator_loss=2.697, discriminator_real_loss=1.464, discriminator_fake_loss=1.232, generator_loss=33.15, generator_mel_loss=25.32, generator_kl_loss=1.94, generator_dur_loss=1.572, generator_adv_loss=2.002, generator_feat_match_loss=2.312, over 100.00 samples.
2024-02-22 15:28:01,167 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 15:30:10,896 INFO [train.py:845] (2/4) Start epoch 99
2024-02-22 15:32:36,397 INFO [train.py:471] (2/4) Epoch 99, batch 24, global_batch_idx: 3650, batch size: 49, loss[discriminator_loss=2.766, discriminator_real_loss=1.468, discriminator_fake_loss=1.298, generator_loss=32, generator_mel_loss=24.37, generator_kl_loss=1.876, generator_dur_loss=1.586, generator_adv_loss=2.023, generator_feat_match_loss=2.145, over 49.00 samples.], tot_loss[discriminator_loss=2.733, discriminator_real_loss=1.392, discriminator_fake_loss=1.342, generator_loss=31.07, generator_mel_loss=23.55, generator_kl_loss=1.928, generator_dur_loss=1.58, generator_adv_loss=1.889, generator_feat_match_loss=2.124, over 1906.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:33:41,986 INFO [train.py:845] (2/4) Start epoch 100
2024-02-22 15:37:16,307 INFO [train.py:845] (2/4) Start epoch 101
2024-02-22 15:37:29,595 INFO [train.py:471] (2/4) Epoch 101, batch 0, global_batch_idx: 3700, batch size: 65, loss[discriminator_loss=2.703, discriminator_real_loss=1.412, discriminator_fake_loss=1.291, generator_loss=31.09, generator_mel_loss=23.4, generator_kl_loss=1.862, generator_dur_loss=1.6, generator_adv_loss=1.953, generator_feat_match_loss=2.277, over 65.00 samples.], tot_loss[discriminator_loss=2.703, discriminator_real_loss=1.412, discriminator_fake_loss=1.291, generator_loss=31.09, generator_mel_loss=23.4, generator_kl_loss=1.862, generator_dur_loss=1.6, generator_adv_loss=1.953, generator_feat_match_loss=2.277, over 65.00 samples.], cur_lr_g: 1.98e-04, cur_lr_d: 1.98e-04, grad_scale: 64.0
2024-02-22 15:40:44,859 INFO [train.py:845] (2/4) Start epoch 102
2024-02-22 15:42:01,478 INFO [train.py:471] (2/4) Epoch 102, batch 13, global_batch_idx: 3750, batch size: 76, loss[discriminator_loss=2.766, discriminator_real_loss=1.517, discriminator_fake_loss=1.25, generator_loss=30.63, generator_mel_loss=23.14, generator_kl_loss=1.83, generator_dur_loss=1.596, generator_adv_loss=1.88, generator_feat_match_loss=2.182, over 76.00 samples.], tot_loss[discriminator_loss=2.749, discriminator_real_loss=1.418, discriminator_fake_loss=1.331, generator_loss=31.1, generator_mel_loss=23.43, generator_kl_loss=1.909, generator_dur_loss=1.579, generator_adv_loss=1.923, generator_feat_match_loss=2.264, over 1045.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 64.0
2024-02-22 15:44:17,629 INFO [train.py:845] (2/4) Start epoch 103
2024-02-22 15:46:38,236 INFO [train.py:471] (2/4) Epoch 103, batch 26, global_batch_idx: 3800, batch size: 126, loss[discriminator_loss=2.719, discriminator_real_loss=1.31, discriminator_fake_loss=1.41, generator_loss=31.06, generator_mel_loss=23.61, generator_kl_loss=1.913, generator_dur_loss=1.578, generator_adv_loss=1.746, generator_feat_match_loss=2.219, over 126.00 samples.], tot_loss[discriminator_loss=2.742, discriminator_real_loss=1.409, discriminator_fake_loss=1.333, generator_loss=30.92, generator_mel_loss=23.33, generator_kl_loss=1.935, generator_dur_loss=1.582, generator_adv_loss=1.894, generator_feat_match_loss=2.173, over 1808.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 64.0
2024-02-22 15:46:38,238 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 15:46:46,721 INFO [train.py:534] (2/4) Epoch 103, validation: discriminator_loss=2.676, discriminator_real_loss=1.214, discriminator_fake_loss=1.462, generator_loss=31.6, generator_mel_loss=24.15, generator_kl_loss=1.974, generator_dur_loss=1.566, generator_adv_loss=1.688, generator_feat_match_loss=2.218, over 100.00 samples.
2024-02-22 15:46:46,722 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 15:47:44,814 INFO [train.py:845] (2/4) Start epoch 104
2024-02-22 15:51:13,832 INFO [train.py:845] (2/4) Start epoch 105
2024-02-22 15:51:42,449 INFO [train.py:471] (2/4) Epoch 105, batch 2, global_batch_idx: 3850, batch size: 95, loss[discriminator_loss=2.771, discriminator_real_loss=1.263, discriminator_fake_loss=1.509, generator_loss=31.3, generator_mel_loss=23.53, generator_kl_loss=1.929, generator_dur_loss=1.551, generator_adv_loss=2.045, generator_feat_match_loss=2.24, over 95.00 samples.], tot_loss[discriminator_loss=2.727, discriminator_real_loss=1.356, discriminator_fake_loss=1.371, generator_loss=31.33, generator_mel_loss=23.57, generator_kl_loss=1.959, generator_dur_loss=1.571, generator_adv_loss=1.961, generator_feat_match_loss=2.267, over 249.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 64.0
2024-02-22 15:54:39,722 INFO [train.py:845] (2/4) Start epoch 106
2024-02-22 15:56:07,136 INFO [train.py:471] (2/4) Epoch 106, batch 15, global_batch_idx: 3900, batch size: 53, loss[discriminator_loss=2.77, discriminator_real_loss=1.26, discriminator_fake_loss=1.509, generator_loss=30.23, generator_mel_loss=22.63, generator_kl_loss=1.835, generator_dur_loss=1.591, generator_adv_loss=2.098, generator_feat_match_loss=2.074, over 53.00 samples.], tot_loss[discriminator_loss=2.75, discriminator_real_loss=1.385, discriminator_fake_loss=1.364, generator_loss=31.31, generator_mel_loss=23.69, generator_kl_loss=1.929, generator_dur_loss=1.581, generator_adv_loss=1.886, generator_feat_match_loss=2.229, over 998.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 64.0
2024-02-22 15:58:03,531 INFO [train.py:845] (2/4) Start epoch 107
2024-02-22 16:00:40,280 INFO [train.py:471] (2/4) Epoch 107, batch 28, global_batch_idx: 3950, batch size: 67, loss[discriminator_loss=2.775, discriminator_real_loss=1.49, discriminator_fake_loss=1.285, generator_loss=30.33, generator_mel_loss=22.74, generator_kl_loss=1.904, generator_dur_loss=1.579, generator_adv_loss=1.994, generator_feat_match_loss=2.109, over 67.00 samples.], tot_loss[discriminator_loss=2.735, discriminator_real_loss=1.405, discriminator_fake_loss=1.33, generator_loss=30.96, generator_mel_loss=23.3, generator_kl_loss=1.933, generator_dur_loss=1.567, generator_adv_loss=1.904, generator_feat_match_loss=2.254, over 2003.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 64.0
2024-02-22 16:01:33,752 INFO [train.py:845] (2/4) Start epoch 108
2024-02-22 16:04:59,684 INFO [train.py:845] (2/4) Start epoch 109
2024-02-22 16:05:33,226 INFO [train.py:471] (2/4) Epoch 109, batch 4, global_batch_idx: 4000, batch size: 73, loss[discriminator_loss=2.785, discriminator_real_loss=1.354, discriminator_fake_loss=1.43, generator_loss=30.32, generator_mel_loss=22.73, generator_kl_loss=1.95, generator_dur_loss=1.562, generator_adv_loss=1.839, generator_feat_match_loss=2.238, over 73.00 samples.], tot_loss[discriminator_loss=2.702, discriminator_real_loss=1.366, discriminator_fake_loss=1.336, generator_loss=30.57, generator_mel_loss=22.9, generator_kl_loss=1.954, generator_dur_loss=1.573, generator_adv_loss=1.87, generator_feat_match_loss=2.276, over 369.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:05:33,228 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 16:05:41,883 INFO [train.py:534] (2/4) Epoch 109, validation: discriminator_loss=2.664, discriminator_real_loss=1.247, discriminator_fake_loss=1.417, generator_loss=31.61, generator_mel_loss=24.09, generator_kl_loss=2.004, generator_dur_loss=1.566, generator_adv_loss=1.779, generator_feat_match_loss=2.168, over 100.00 samples.
2024-02-22 16:05:41,887 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 16:08:40,815 INFO [train.py:845] (2/4) Start epoch 110
2024-02-22 16:10:24,761 INFO [train.py:471] (2/4) Epoch 110, batch 17, global_batch_idx: 4050, batch size: 52, loss[discriminator_loss=2.691, discriminator_real_loss=1.186, discriminator_fake_loss=1.505, generator_loss=30.43, generator_mel_loss=22.61, generator_kl_loss=1.867, generator_dur_loss=1.574, generator_adv_loss=2.193, generator_feat_match_loss=2.189, over 52.00 samples.], tot_loss[discriminator_loss=2.715, discriminator_real_loss=1.375, discriminator_fake_loss=1.34, generator_loss=30.86, generator_mel_loss=23.19, generator_kl_loss=1.946, generator_dur_loss=1.562, generator_adv_loss=1.899, generator_feat_match_loss=2.27, over 1408.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:12:10,293 INFO [train.py:845] (2/4) Start epoch 111
2024-02-22 16:15:10,974 INFO [train.py:471] (2/4) Epoch 111, batch 30, global_batch_idx: 4100, batch size: 71, loss[discriminator_loss=2.723, discriminator_real_loss=1.312, discriminator_fake_loss=1.412, generator_loss=31.68, generator_mel_loss=23.68, generator_kl_loss=1.992, generator_dur_loss=1.562, generator_adv_loss=2.068, generator_feat_match_loss=2.377, over 71.00 samples.], tot_loss[discriminator_loss=2.728, discriminator_real_loss=1.385, discriminator_fake_loss=1.343, generator_loss=30.88, generator_mel_loss=23.21, generator_kl_loss=1.945, generator_dur_loss=1.569, generator_adv_loss=1.888, generator_feat_match_loss=2.269, over 2176.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:15:41,852 INFO [train.py:845] (2/4) Start epoch 112
2024-02-22 16:19:13,478 INFO [train.py:845] (2/4) Start epoch 113
2024-02-22 16:19:56,117 INFO [train.py:471] (2/4) Epoch 113, batch 6, global_batch_idx: 4150, batch size: 85, loss[discriminator_loss=2.676, discriminator_real_loss=1.47, discriminator_fake_loss=1.206, generator_loss=30.52, generator_mel_loss=22.84, generator_kl_loss=1.971, generator_dur_loss=1.575, generator_adv_loss=1.715, generator_feat_match_loss=2.42, over 85.00 samples.], tot_loss[discriminator_loss=2.695, discriminator_real_loss=1.381, discriminator_fake_loss=1.314, generator_loss=30.75, generator_mel_loss=23.06, generator_kl_loss=1.947, generator_dur_loss=1.572, generator_adv_loss=1.878, generator_feat_match_loss=2.29, over 464.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:22:37,733 INFO [train.py:845] (2/4) Start epoch 114
2024-02-22 16:24:38,572 INFO [train.py:471] (2/4) Epoch 114, batch 19, global_batch_idx: 4200, batch size: 101, loss[discriminator_loss=2.68, discriminator_real_loss=1.344, discriminator_fake_loss=1.336, generator_loss=31.41, generator_mel_loss=23.45, generator_kl_loss=1.972, generator_dur_loss=1.546, generator_adv_loss=1.923, generator_feat_match_loss=2.52, over 101.00 samples.], tot_loss[discriminator_loss=2.708, discriminator_real_loss=1.372, discriminator_fake_loss=1.336, generator_loss=31.01, generator_mel_loss=23.22, generator_kl_loss=1.933, generator_dur_loss=1.557, generator_adv_loss=1.907, generator_feat_match_loss=2.387, over 1664.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:24:38,574 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 16:24:46,974 INFO [train.py:534] (2/4) Epoch 114, validation: discriminator_loss=2.712, discriminator_real_loss=1.322, discriminator_fake_loss=1.39, generator_loss=32.6, generator_mel_loss=24.62, generator_kl_loss=2.024, generator_dur_loss=1.562, generator_adv_loss=1.793, generator_feat_match_loss=2.605, over 100.00 samples.
2024-02-22 16:24:46,974 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 16:26:11,330 INFO [train.py:845] (2/4) Start epoch 115
2024-02-22 16:29:16,207 INFO [train.py:471] (2/4) Epoch 115, batch 32, global_batch_idx: 4250, batch size: 85, loss[discriminator_loss=2.689, discriminator_real_loss=1.324, discriminator_fake_loss=1.365, generator_loss=30.73, generator_mel_loss=23.11, generator_kl_loss=1.782, generator_dur_loss=1.558, generator_adv_loss=1.994, generator_feat_match_loss=2.279, over 85.00 samples.], tot_loss[discriminator_loss=2.751, discriminator_real_loss=1.388, discriminator_fake_loss=1.363, generator_loss=30.82, generator_mel_loss=23.16, generator_kl_loss=1.921, generator_dur_loss=1.562, generator_adv_loss=1.914, generator_feat_match_loss=2.263, over 2505.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:29:38,734 INFO [train.py:845] (2/4) Start epoch 116
2024-02-22 16:33:01,098 INFO [train.py:845] (2/4) Start epoch 117
2024-02-22 16:33:58,379 INFO [train.py:471] (2/4) Epoch 117, batch 8, global_batch_idx: 4300, batch size: 63, loss[discriminator_loss=2.656, discriminator_real_loss=1.381, discriminator_fake_loss=1.275, generator_loss=30.73, generator_mel_loss=23.05, generator_kl_loss=1.851, generator_dur_loss=1.582, generator_adv_loss=1.874, generator_feat_match_loss=2.375, over 63.00 samples.], tot_loss[discriminator_loss=2.707, discriminator_real_loss=1.375, discriminator_fake_loss=1.332, generator_loss=30.76, generator_mel_loss=23.03, generator_kl_loss=1.927, generator_dur_loss=1.561, generator_adv_loss=1.919, generator_feat_match_loss=2.322, over 668.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:36:35,872 INFO [train.py:845] (2/4) Start epoch 118
2024-02-22 16:38:48,977 INFO [train.py:471] (2/4) Epoch 118, batch 21, global_batch_idx: 4350, batch size: 126, loss[discriminator_loss=2.754, discriminator_real_loss=1.244, discriminator_fake_loss=1.51, generator_loss=31.23, generator_mel_loss=23.37, generator_kl_loss=1.911, generator_dur_loss=1.537, generator_adv_loss=2.061, generator_feat_match_loss=2.352, over 126.00 samples.], tot_loss[discriminator_loss=2.723, discriminator_real_loss=1.385, discriminator_fake_loss=1.337, generator_loss=30.86, generator_mel_loss=23.13, generator_kl_loss=1.908, generator_dur_loss=1.548, generator_adv_loss=1.907, generator_feat_match_loss=2.366, over 2017.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:40:06,409 INFO [train.py:845] (2/4) Start epoch 119
2024-02-22 16:43:28,117 INFO [train.py:471] (2/4) Epoch 119, batch 34, global_batch_idx: 4400, batch size: 126, loss[discriminator_loss=2.699, discriminator_real_loss=1.512, discriminator_fake_loss=1.188, generator_loss=30.84, generator_mel_loss=23.19, generator_kl_loss=1.858, generator_dur_loss=1.532, generator_adv_loss=1.777, generator_feat_match_loss=2.48, over 126.00 samples.], tot_loss[discriminator_loss=2.731, discriminator_real_loss=1.406, discriminator_fake_loss=1.325, generator_loss=30.74, generator_mel_loss=23.1, generator_kl_loss=1.91, generator_dur_loss=1.553, generator_adv_loss=1.873, generator_feat_match_loss=2.303, over 2695.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:43:28,118 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 16:43:36,718 INFO [train.py:534] (2/4) Epoch 119, validation: discriminator_loss=2.647, discriminator_real_loss=1.239, discriminator_fake_loss=1.408, generator_loss=31.25, generator_mel_loss=23.6, generator_kl_loss=1.934, generator_dur_loss=1.557, generator_adv_loss=1.718, generator_feat_match_loss=2.443, over 100.00 samples.
2024-02-22 16:43:36,719 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 16:43:46,638 INFO [train.py:845] (2/4) Start epoch 120
2024-02-22 16:47:15,388 INFO [train.py:845] (2/4) Start epoch 121
2024-02-22 16:48:24,281 INFO [train.py:471] (2/4) Epoch 121, batch 10, global_batch_idx: 4450, batch size: 153, loss[discriminator_loss=2.676, discriminator_real_loss=1.296, discriminator_fake_loss=1.379, generator_loss=31.54, generator_mel_loss=23.78, generator_kl_loss=1.92, generator_dur_loss=1.518, generator_adv_loss=1.94, generator_feat_match_loss=2.381, over 153.00 samples.], tot_loss[discriminator_loss=2.748, discriminator_real_loss=1.385, discriminator_fake_loss=1.363, generator_loss=30.76, generator_mel_loss=23.17, generator_kl_loss=1.893, generator_dur_loss=1.552, generator_adv_loss=1.89, generator_feat_match_loss=2.254, over 876.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:50:46,931 INFO [train.py:845] (2/4) Start epoch 122
2024-02-22 16:53:06,841 INFO [train.py:471] (2/4) Epoch 122, batch 23, global_batch_idx: 4500, batch size: 76, loss[discriminator_loss=2.723, discriminator_real_loss=1.49, discriminator_fake_loss=1.232, generator_loss=30.78, generator_mel_loss=23.13, generator_kl_loss=1.887, generator_dur_loss=1.548, generator_adv_loss=1.853, generator_feat_match_loss=2.365, over 76.00 samples.], tot_loss[discriminator_loss=2.73, discriminator_real_loss=1.397, discriminator_fake_loss=1.333, generator_loss=30.69, generator_mel_loss=23.01, generator_kl_loss=1.893, generator_dur_loss=1.555, generator_adv_loss=1.916, generator_feat_match_loss=2.322, over 1719.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:54:14,804 INFO [train.py:845] (2/4) Start epoch 123
2024-02-22 16:57:38,787 INFO [train.py:471] (2/4) Epoch 123, batch 36, global_batch_idx: 4550, batch size: 85, loss[discriminator_loss=2.734, discriminator_real_loss=1.403, discriminator_fake_loss=1.331, generator_loss=31.25, generator_mel_loss=23.19, generator_kl_loss=1.929, generator_dur_loss=1.559, generator_adv_loss=2.158, generator_feat_match_loss=2.418, over 85.00 samples.], tot_loss[discriminator_loss=2.732, discriminator_real_loss=1.398, discriminator_fake_loss=1.334, generator_loss=30.66, generator_mel_loss=22.96, generator_kl_loss=1.923, generator_dur_loss=1.553, generator_adv_loss=1.9, generator_feat_match_loss=2.323, over 2679.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 16:57:39,211 INFO [train.py:845] (2/4) Start epoch 124
2024-02-22 17:01:06,734 INFO [train.py:845] (2/4) Start epoch 125
2024-02-22 17:02:30,277 INFO [train.py:471] (2/4) Epoch 125, batch 12, global_batch_idx: 4600, batch size: 69, loss[discriminator_loss=2.74, discriminator_real_loss=1.311, discriminator_fake_loss=1.43, generator_loss=30.74, generator_mel_loss=22.75, generator_kl_loss=2.009, generator_dur_loss=1.547, generator_adv_loss=2.207, generator_feat_match_loss=2.221, over 69.00 samples.], tot_loss[discriminator_loss=2.735, discriminator_real_loss=1.391, discriminator_fake_loss=1.343, generator_loss=30.66, generator_mel_loss=22.93, generator_kl_loss=1.918, generator_dur_loss=1.553, generator_adv_loss=1.923, generator_feat_match_loss=2.336, over 975.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:02:30,279 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 17:02:38,867 INFO [train.py:534] (2/4) Epoch 125, validation: discriminator_loss=2.717, discriminator_real_loss=1.458, discriminator_fake_loss=1.259, generator_loss=32.11, generator_mel_loss=24.07, generator_kl_loss=1.999, generator_dur_loss=1.546, generator_adv_loss=2.02, generator_feat_match_loss=2.477, over 100.00 samples.
2024-02-22 17:02:38,868 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 17:04:48,109 INFO [train.py:845] (2/4) Start epoch 126
2024-02-22 17:07:12,607 INFO [train.py:471] (2/4) Epoch 126, batch 25, global_batch_idx: 4650, batch size: 71, loss[discriminator_loss=2.832, discriminator_real_loss=1.353, discriminator_fake_loss=1.479, generator_loss=30.98, generator_mel_loss=22.73, generator_kl_loss=1.96, generator_dur_loss=1.56, generator_adv_loss=2.438, generator_feat_match_loss=2.291, over 71.00 samples.], tot_loss[discriminator_loss=2.737, discriminator_real_loss=1.402, discriminator_fake_loss=1.335, generator_loss=30.89, generator_mel_loss=23.04, generator_kl_loss=1.95, generator_dur_loss=1.56, generator_adv_loss=1.935, generator_feat_match_loss=2.409, over 1883.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:08:11,813 INFO [train.py:845] (2/4) Start epoch 127
2024-02-22 17:11:46,559 INFO [train.py:845] (2/4) Start epoch 128
2024-02-22 17:12:09,129 INFO [train.py:471] (2/4) Epoch 128, batch 1, global_batch_idx: 4700, batch size: 101, loss[discriminator_loss=2.762, discriminator_real_loss=1.562, discriminator_fake_loss=1.2, generator_loss=30.42, generator_mel_loss=22.9, generator_kl_loss=1.889, generator_dur_loss=1.556, generator_adv_loss=1.901, generator_feat_match_loss=2.174, over 101.00 samples.], tot_loss[discriminator_loss=2.778, discriminator_real_loss=1.502, discriminator_fake_loss=1.275, generator_loss=30.39, generator_mel_loss=22.85, generator_kl_loss=1.899, generator_dur_loss=1.557, generator_adv_loss=1.947, generator_feat_match_loss=2.137, over 160.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:15:20,463 INFO [train.py:845] (2/4) Start epoch 129
2024-02-22 17:16:47,571 INFO [train.py:471] (2/4) Epoch 129, batch 14, global_batch_idx: 4750, batch size: 95, loss[discriminator_loss=2.746, discriminator_real_loss=1.47, discriminator_fake_loss=1.275, generator_loss=31.13, generator_mel_loss=23.36, generator_kl_loss=1.913, generator_dur_loss=1.553, generator_adv_loss=1.912, generator_feat_match_loss=2.398, over 95.00 samples.], tot_loss[discriminator_loss=2.739, discriminator_real_loss=1.388, discriminator_fake_loss=1.351, generator_loss=30.6, generator_mel_loss=22.87, generator_kl_loss=1.908, generator_dur_loss=1.555, generator_adv_loss=1.925, generator_feat_match_loss=2.347, over 1053.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:18:52,283 INFO [train.py:845] (2/4) Start epoch 130
2024-02-22 17:21:28,610 INFO [train.py:471] (2/4) Epoch 130, batch 27, global_batch_idx: 4800, batch size: 65, loss[discriminator_loss=2.676, discriminator_real_loss=1.337, discriminator_fake_loss=1.338, generator_loss=29.75, generator_mel_loss=22.01, generator_kl_loss=1.813, generator_dur_loss=1.55, generator_adv_loss=2.01, generator_feat_match_loss=2.367, over 65.00 samples.], tot_loss[discriminator_loss=2.758, discriminator_real_loss=1.409, discriminator_fake_loss=1.349, generator_loss=30.44, generator_mel_loss=22.79, generator_kl_loss=1.906, generator_dur_loss=1.557, generator_adv_loss=1.893, generator_feat_match_loss=2.29, over 1946.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:21:28,611 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 17:21:37,064 INFO [train.py:534] (2/4) Epoch 130, validation: discriminator_loss=2.623, discriminator_real_loss=1.363, discriminator_fake_loss=1.26, generator_loss=31.27, generator_mel_loss=23.43, generator_kl_loss=1.931, generator_dur_loss=1.541, generator_adv_loss=1.919, generator_feat_match_loss=2.456, over 100.00 samples.
2024-02-22 17:21:37,065 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 17:22:26,878 INFO [train.py:845] (2/4) Start epoch 131
2024-02-22 17:25:55,259 INFO [train.py:845] (2/4) Start epoch 132
2024-02-22 17:26:26,995 INFO [train.py:471] (2/4) Epoch 132, batch 3, global_batch_idx: 4850, batch size: 59, loss[discriminator_loss=2.727, discriminator_real_loss=1.468, discriminator_fake_loss=1.258, generator_loss=31.35, generator_mel_loss=23.6, generator_kl_loss=2.038, generator_dur_loss=1.566, generator_adv_loss=1.829, generator_feat_match_loss=2.32, over 59.00 samples.], tot_loss[discriminator_loss=2.746, discriminator_real_loss=1.401, discriminator_fake_loss=1.345, generator_loss=30.7, generator_mel_loss=23.03, generator_kl_loss=1.971, generator_dur_loss=1.572, generator_adv_loss=1.84, generator_feat_match_loss=2.296, over 238.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:29:22,356 INFO [train.py:845] (2/4) Start epoch 133
2024-02-22 17:30:56,761 INFO [train.py:471] (2/4) Epoch 133, batch 16, global_batch_idx: 4900, batch size: 71, loss[discriminator_loss=2.691, discriminator_real_loss=1.378, discriminator_fake_loss=1.312, generator_loss=31.33, generator_mel_loss=23.56, generator_kl_loss=1.953, generator_dur_loss=1.551, generator_adv_loss=1.852, generator_feat_match_loss=2.416, over 71.00 samples.], tot_loss[discriminator_loss=2.743, discriminator_real_loss=1.401, discriminator_fake_loss=1.342, generator_loss=30.43, generator_mel_loss=22.79, generator_kl_loss=1.94, generator_dur_loss=1.55, generator_adv_loss=1.895, generator_feat_match_loss=2.249, over 1095.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:32:49,912 INFO [train.py:845] (2/4) Start epoch 134
2024-02-22 17:35:37,061 INFO [train.py:471] (2/4) Epoch 134, batch 29, global_batch_idx: 4950, batch size: 90, loss[discriminator_loss=2.73, discriminator_real_loss=1.314, discriminator_fake_loss=1.416, generator_loss=30.42, generator_mel_loss=22.62, generator_kl_loss=1.898, generator_dur_loss=1.555, generator_adv_loss=1.922, generator_feat_match_loss=2.43, over 90.00 samples.], tot_loss[discriminator_loss=2.725, discriminator_real_loss=1.394, discriminator_fake_loss=1.331, generator_loss=30.56, generator_mel_loss=22.79, generator_kl_loss=1.922, generator_dur_loss=1.546, generator_adv_loss=1.894, generator_feat_match_loss=2.403, over 2293.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:36:15,979 INFO [train.py:845] (2/4) Start epoch 135
2024-02-22 17:39:41,753 INFO [train.py:845] (2/4) Start epoch 136
2024-02-22 17:40:21,127 INFO [train.py:471] (2/4) Epoch 136, batch 5, global_batch_idx: 5000, batch size: 52, loss[discriminator_loss=2.668, discriminator_real_loss=1.462, discriminator_fake_loss=1.207, generator_loss=30.66, generator_mel_loss=22.67, generator_kl_loss=1.975, generator_dur_loss=1.555, generator_adv_loss=1.752, generator_feat_match_loss=2.707, over 52.00 samples.], tot_loss[discriminator_loss=2.702, discriminator_real_loss=1.407, discriminator_fake_loss=1.296, generator_loss=30.36, generator_mel_loss=22.52, generator_kl_loss=1.924, generator_dur_loss=1.541, generator_adv_loss=1.922, generator_feat_match_loss=2.451, over 393.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:40:21,129 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 17:40:30,078 INFO [train.py:534] (2/4) Epoch 136, validation: discriminator_loss=2.765, discriminator_real_loss=1.236, discriminator_fake_loss=1.53, generator_loss=31.5, generator_mel_loss=23.68, generator_kl_loss=2.09, generator_dur_loss=1.539, generator_adv_loss=1.613, generator_feat_match_loss=2.586, over 100.00 samples.
2024-02-22 17:40:30,080 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 17:43:14,231 INFO [train.py:845] (2/4) Start epoch 137
2024-02-22 17:45:06,360 INFO [train.py:471] (2/4) Epoch 137, batch 18, global_batch_idx: 5050, batch size: 54, loss[discriminator_loss=2.812, discriminator_real_loss=1.423, discriminator_fake_loss=1.391, generator_loss=29.95, generator_mel_loss=22.38, generator_kl_loss=1.926, generator_dur_loss=1.565, generator_adv_loss=1.889, generator_feat_match_loss=2.189, over 54.00 samples.], tot_loss[discriminator_loss=2.729, discriminator_real_loss=1.401, discriminator_fake_loss=1.328, generator_loss=30.42, generator_mel_loss=22.72, generator_kl_loss=1.929, generator_dur_loss=1.548, generator_adv_loss=1.888, generator_feat_match_loss=2.334, over 1465.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:46:45,601 INFO [train.py:845] (2/4) Start epoch 138
2024-02-22 17:49:50,528 INFO [train.py:471] (2/4) Epoch 138, batch 31, global_batch_idx: 5100, batch size: 67, loss[discriminator_loss=2.719, discriminator_real_loss=1.467, discriminator_fake_loss=1.251, generator_loss=30.94, generator_mel_loss=23.19, generator_kl_loss=1.883, generator_dur_loss=1.557, generator_adv_loss=1.84, generator_feat_match_loss=2.469, over 67.00 samples.], tot_loss[discriminator_loss=2.733, discriminator_real_loss=1.396, discriminator_fake_loss=1.337, generator_loss=30.41, generator_mel_loss=22.64, generator_kl_loss=1.907, generator_dur_loss=1.551, generator_adv_loss=1.904, generator_feat_match_loss=2.399, over 2260.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:50:18,427 INFO [train.py:845] (2/4) Start epoch 139
2024-02-22 17:53:43,306 INFO [train.py:845] (2/4) Start epoch 140
2024-02-22 17:54:33,535 INFO [train.py:471] (2/4) Epoch 140, batch 7, global_batch_idx: 5150, batch size: 81, loss[discriminator_loss=2.721, discriminator_real_loss=1.408, discriminator_fake_loss=1.312, generator_loss=30.98, generator_mel_loss=23.18, generator_kl_loss=1.95, generator_dur_loss=1.529, generator_adv_loss=1.799, generator_feat_match_loss=2.523, over 81.00 samples.], tot_loss[discriminator_loss=2.761, discriminator_real_loss=1.405, discriminator_fake_loss=1.356, generator_loss=30.45, generator_mel_loss=22.76, generator_kl_loss=1.901, generator_dur_loss=1.556, generator_adv_loss=1.88, generator_feat_match_loss=2.352, over 465.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:57:17,902 INFO [train.py:845] (2/4) Start epoch 141
2024-02-22 17:59:20,202 INFO [train.py:471] (2/4) Epoch 141, batch 20, global_batch_idx: 5200, batch size: 67, loss[discriminator_loss=2.805, discriminator_real_loss=1.24, discriminator_fake_loss=1.563, generator_loss=30.47, generator_mel_loss=22.53, generator_kl_loss=1.968, generator_dur_loss=1.559, generator_adv_loss=2.047, generator_feat_match_loss=2.371, over 67.00 samples.], tot_loss[discriminator_loss=2.749, discriminator_real_loss=1.399, discriminator_fake_loss=1.35, generator_loss=30.2, generator_mel_loss=22.53, generator_kl_loss=1.915, generator_dur_loss=1.551, generator_adv_loss=1.872, generator_feat_match_loss=2.332, over 1449.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 17:59:20,203 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 17:59:28,678 INFO [train.py:534] (2/4) Epoch 141, validation: discriminator_loss=2.677, discriminator_real_loss=1.462, discriminator_fake_loss=1.214, generator_loss=31.78, generator_mel_loss=23.67, generator_kl_loss=2.071, generator_dur_loss=1.541, generator_adv_loss=1.969, generator_feat_match_loss=2.528, over 100.00 samples.
2024-02-22 17:59:28,679 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 18:00:49,767 INFO [train.py:845] (2/4) Start epoch 142
2024-02-22 18:03:58,582 INFO [train.py:471] (2/4) Epoch 142, batch 33, global_batch_idx: 5250, batch size: 95, loss[discriminator_loss=2.75, discriminator_real_loss=1.356, discriminator_fake_loss=1.394, generator_loss=29.99, generator_mel_loss=22.45, generator_kl_loss=1.86, generator_dur_loss=1.517, generator_adv_loss=1.93, generator_feat_match_loss=2.234, over 95.00 samples.], tot_loss[discriminator_loss=2.766, discriminator_real_loss=1.411, discriminator_fake_loss=1.355, generator_loss=30.45, generator_mel_loss=22.65, generator_kl_loss=1.933, generator_dur_loss=1.542, generator_adv_loss=1.925, generator_feat_match_loss=2.403, over 2509.00 samples.], cur_lr_g: 1.97e-04, cur_lr_d: 1.97e-04, grad_scale: 128.0
2024-02-22 18:04:16,211 INFO [train.py:845] (2/4) Start epoch 143
2024-02-22 18:07:41,608 INFO [train.py:845] (2/4) Start epoch 144
2024-02-22 18:08:48,237 INFO [train.py:471] (2/4) Epoch 144, batch 9, global_batch_idx: 5300, batch size: 76, loss[discriminator_loss=2.695, discriminator_real_loss=1.243, discriminator_fake_loss=1.452, generator_loss=31.12, generator_mel_loss=23.13, generator_kl_loss=1.973, generator_dur_loss=1.557, generator_adv_loss=1.977, generator_feat_match_loss=2.475, over 76.00 samples.], tot_loss[discriminator_loss=2.727, discriminator_real_loss=1.38, discriminator_fake_loss=1.347, generator_loss=30.43, generator_mel_loss=22.68, generator_kl_loss=1.927, generator_dur_loss=1.542, generator_adv_loss=1.879, generator_feat_match_loss=2.397, over 719.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:11:14,631 INFO [train.py:845] (2/4) Start epoch 145
2024-02-22 18:13:26,100 INFO [train.py:471] (2/4) Epoch 145, batch 22, global_batch_idx: 5350, batch size: 110, loss[discriminator_loss=2.766, discriminator_real_loss=1.412, discriminator_fake_loss=1.354, generator_loss=30.24, generator_mel_loss=22.61, generator_kl_loss=1.888, generator_dur_loss=1.527, generator_adv_loss=1.796, generator_feat_match_loss=2.422, over 110.00 samples.], tot_loss[discriminator_loss=2.722, discriminator_real_loss=1.391, discriminator_fake_loss=1.331, generator_loss=30.27, generator_mel_loss=22.48, generator_kl_loss=1.896, generator_dur_loss=1.541, generator_adv_loss=1.897, generator_feat_match_loss=2.447, over 1732.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:14:46,130 INFO [train.py:845] (2/4) Start epoch 146
2024-02-22 18:18:09,057 INFO [train.py:471] (2/4) Epoch 146, batch 35, global_batch_idx: 5400, batch size: 69, loss[discriminator_loss=2.697, discriminator_real_loss=1.452, discriminator_fake_loss=1.245, generator_loss=30.4, generator_mel_loss=22.71, generator_kl_loss=1.928, generator_dur_loss=1.526, generator_adv_loss=1.867, generator_feat_match_loss=2.371, over 69.00 samples.], tot_loss[discriminator_loss=2.745, discriminator_real_loss=1.399, discriminator_fake_loss=1.346, generator_loss=30.38, generator_mel_loss=22.59, generator_kl_loss=1.919, generator_dur_loss=1.543, generator_adv_loss=1.926, generator_feat_match_loss=2.399, over 2754.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:18:09,058 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 18:18:19,168 INFO [train.py:534] (2/4) Epoch 146, validation: discriminator_loss=2.684, discriminator_real_loss=1.228, discriminator_fake_loss=1.456, generator_loss=31.15, generator_mel_loss=23.34, generator_kl_loss=1.962, generator_dur_loss=1.539, generator_adv_loss=1.713, generator_feat_match_loss=2.603, over 100.00 samples.
2024-02-22 18:18:19,169 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 18:18:24,271 INFO [train.py:845] (2/4) Start epoch 147
2024-02-22 18:21:51,142 INFO [train.py:845] (2/4) Start epoch 148
2024-02-22 18:22:58,065 INFO [train.py:471] (2/4) Epoch 148, batch 11, global_batch_idx: 5450, batch size: 81, loss[discriminator_loss=2.713, discriminator_real_loss=1.416, discriminator_fake_loss=1.297, generator_loss=30.45, generator_mel_loss=22.57, generator_kl_loss=1.949, generator_dur_loss=1.531, generator_adv_loss=1.846, generator_feat_match_loss=2.555, over 81.00 samples.], tot_loss[discriminator_loss=2.725, discriminator_real_loss=1.381, discriminator_fake_loss=1.344, generator_loss=30.34, generator_mel_loss=22.51, generator_kl_loss=1.92, generator_dur_loss=1.539, generator_adv_loss=1.911, generator_feat_match_loss=2.468, over 773.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:25:17,241 INFO [train.py:845] (2/4) Start epoch 149
2024-02-22 18:27:41,923 INFO [train.py:471] (2/4) Epoch 149, batch 24, global_batch_idx: 5500, batch size: 51, loss[discriminator_loss=2.691, discriminator_real_loss=1.471, discriminator_fake_loss=1.221, generator_loss=30.5, generator_mel_loss=22.51, generator_kl_loss=1.951, generator_dur_loss=1.567, generator_adv_loss=1.994, generator_feat_match_loss=2.48, over 51.00 samples.], tot_loss[discriminator_loss=2.726, discriminator_real_loss=1.397, discriminator_fake_loss=1.329, generator_loss=30.36, generator_mel_loss=22.57, generator_kl_loss=1.898, generator_dur_loss=1.54, generator_adv_loss=1.899, generator_feat_match_loss=2.451, over 1806.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:28:48,711 INFO [train.py:845] (2/4) Start epoch 150
2024-02-22 18:32:12,608 INFO [train.py:845] (2/4) Start epoch 151
2024-02-22 18:32:24,592 INFO [train.py:471] (2/4) Epoch 151, batch 0, global_batch_idx: 5550, batch size: 50, loss[discriminator_loss=2.789, discriminator_real_loss=1.411, discriminator_fake_loss=1.378, generator_loss=30.24, generator_mel_loss=22.48, generator_kl_loss=1.899, generator_dur_loss=1.555, generator_adv_loss=1.885, generator_feat_match_loss=2.416, over 50.00 samples.], tot_loss[discriminator_loss=2.789, discriminator_real_loss=1.411, discriminator_fake_loss=1.378, generator_loss=30.24, generator_mel_loss=22.48, generator_kl_loss=1.899, generator_dur_loss=1.555, generator_adv_loss=1.885, generator_feat_match_loss=2.416, over 50.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:35:39,333 INFO [train.py:845] (2/4) Start epoch 152
2024-02-22 18:36:57,596 INFO [train.py:471] (2/4) Epoch 152, batch 13, global_batch_idx: 5600, batch size: 56, loss[discriminator_loss=2.75, discriminator_real_loss=1.278, discriminator_fake_loss=1.473, generator_loss=29.62, generator_mel_loss=21.81, generator_kl_loss=1.856, generator_dur_loss=1.568, generator_adv_loss=2.133, generator_feat_match_loss=2.25, over 56.00 samples.], tot_loss[discriminator_loss=2.697, discriminator_real_loss=1.373, discriminator_fake_loss=1.325, generator_loss=30.45, generator_mel_loss=22.55, generator_kl_loss=1.886, generator_dur_loss=1.542, generator_adv_loss=1.937, generator_feat_match_loss=2.531, over 968.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:36:57,598 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 18:37:06,543 INFO [train.py:534] (2/4) Epoch 152, validation: discriminator_loss=2.756, discriminator_real_loss=1.49, discriminator_fake_loss=1.266, generator_loss=31.42, generator_mel_loss=23.29, generator_kl_loss=1.951, generator_dur_loss=1.531, generator_adv_loss=2.054, generator_feat_match_loss=2.602, over 100.00 samples.
2024-02-22 18:37:06,544 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 18:39:12,206 INFO [train.py:845] (2/4) Start epoch 153
2024-02-22 18:41:47,661 INFO [train.py:471] (2/4) Epoch 153, batch 26, global_batch_idx: 5650, batch size: 79, loss[discriminator_loss=2.742, discriminator_real_loss=1.321, discriminator_fake_loss=1.42, generator_loss=29.52, generator_mel_loss=21.85, generator_kl_loss=1.917, generator_dur_loss=1.543, generator_adv_loss=1.83, generator_feat_match_loss=2.381, over 79.00 samples.], tot_loss[discriminator_loss=2.741, discriminator_real_loss=1.39, discriminator_fake_loss=1.352, generator_loss=30.28, generator_mel_loss=22.47, generator_kl_loss=1.9, generator_dur_loss=1.543, generator_adv_loss=1.911, generator_feat_match_loss=2.451, over 1997.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:42:45,297 INFO [train.py:845] (2/4) Start epoch 154
2024-02-22 18:46:10,328 INFO [train.py:845] (2/4) Start epoch 155
2024-02-22 18:46:36,330 INFO [train.py:471] (2/4) Epoch 155, batch 2, global_batch_idx: 5700, batch size: 153, loss[discriminator_loss=2.707, discriminator_real_loss=1.291, discriminator_fake_loss=1.416, generator_loss=30.71, generator_mel_loss=22.66, generator_kl_loss=1.918, generator_dur_loss=1.534, generator_adv_loss=2.043, generator_feat_match_loss=2.551, over 153.00 samples.], tot_loss[discriminator_loss=2.71, discriminator_real_loss=1.33, discriminator_fake_loss=1.38, generator_loss=30.27, generator_mel_loss=22.34, generator_kl_loss=1.901, generator_dur_loss=1.543, generator_adv_loss=1.975, generator_feat_match_loss=2.506, over 254.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:49:43,759 INFO [train.py:845] (2/4) Start epoch 156
2024-02-22 18:51:20,100 INFO [train.py:471] (2/4) Epoch 156, batch 15, global_batch_idx: 5750, batch size: 49, loss[discriminator_loss=2.719, discriminator_real_loss=1.415, discriminator_fake_loss=1.305, generator_loss=30.38, generator_mel_loss=22.51, generator_kl_loss=1.845, generator_dur_loss=1.586, generator_adv_loss=2.094, generator_feat_match_loss=2.346, over 49.00 samples.], tot_loss[discriminator_loss=2.717, discriminator_real_loss=1.382, discriminator_fake_loss=1.336, generator_loss=30.16, generator_mel_loss=22.35, generator_kl_loss=1.91, generator_dur_loss=1.541, generator_adv_loss=1.903, generator_feat_match_loss=2.457, over 1127.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:53:17,238 INFO [train.py:845] (2/4) Start epoch 157
2024-02-22 18:56:06,301 INFO [train.py:471] (2/4) Epoch 157, batch 28, global_batch_idx: 5800, batch size: 81, loss[discriminator_loss=2.715, discriminator_real_loss=1.272, discriminator_fake_loss=1.443, generator_loss=30.83, generator_mel_loss=22.82, generator_kl_loss=1.927, generator_dur_loss=1.535, generator_adv_loss=1.967, generator_feat_match_loss=2.59, over 81.00 samples.], tot_loss[discriminator_loss=2.733, discriminator_real_loss=1.385, discriminator_fake_loss=1.349, generator_loss=30.1, generator_mel_loss=22.37, generator_kl_loss=1.916, generator_dur_loss=1.542, generator_adv_loss=1.859, generator_feat_match_loss=2.419, over 2022.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 18:56:06,302 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 18:56:14,940 INFO [train.py:534] (2/4) Epoch 157, validation: discriminator_loss=2.62, discriminator_real_loss=1.354, discriminator_fake_loss=1.267, generator_loss=30.58, generator_mel_loss=22.7, generator_kl_loss=1.914, generator_dur_loss=1.532, generator_adv_loss=1.954, generator_feat_match_loss=2.476, over 100.00 samples.
2024-02-22 18:56:14,941 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 18:56:53,329 INFO [train.py:845] (2/4) Start epoch 158
2024-02-22 19:00:21,680 INFO [train.py:845] (2/4) Start epoch 159
2024-02-22 19:01:00,212 INFO [train.py:471] (2/4) Epoch 159, batch 4, global_batch_idx: 5850, batch size: 153, loss[discriminator_loss=2.75, discriminator_real_loss=1.446, discriminator_fake_loss=1.304, generator_loss=29.77, generator_mel_loss=22.3, generator_kl_loss=1.876, generator_dur_loss=1.506, generator_adv_loss=1.789, generator_feat_match_loss=2.297, over 153.00 samples.], tot_loss[discriminator_loss=2.786, discriminator_real_loss=1.428, discriminator_fake_loss=1.358, generator_loss=29.82, generator_mel_loss=22.28, generator_kl_loss=1.881, generator_dur_loss=1.523, generator_adv_loss=1.844, generator_feat_match_loss=2.301, over 448.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 19:03:47,350 INFO [train.py:845] (2/4) Start epoch 160
2024-02-22 19:05:36,078 INFO [train.py:471] (2/4) Epoch 160, batch 17, global_batch_idx: 5900, batch size: 67, loss[discriminator_loss=2.754, discriminator_real_loss=1.276, discriminator_fake_loss=1.477, generator_loss=30.36, generator_mel_loss=22.26, generator_kl_loss=1.988, generator_dur_loss=1.522, generator_adv_loss=2.1, generator_feat_match_loss=2.484, over 67.00 samples.], tot_loss[discriminator_loss=2.721, discriminator_real_loss=1.376, discriminator_fake_loss=1.345, generator_loss=30.12, generator_mel_loss=22.21, generator_kl_loss=1.898, generator_dur_loss=1.535, generator_adv_loss=1.932, generator_feat_match_loss=2.546, over 1336.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 19:07:18,179 INFO [train.py:845] (2/4) Start epoch 161
2024-02-22 19:10:19,405 INFO [train.py:471] (2/4) Epoch 161, batch 30, global_batch_idx: 5950, batch size: 51, loss[discriminator_loss=2.762, discriminator_real_loss=1.493, discriminator_fake_loss=1.269, generator_loss=28.82, generator_mel_loss=21.46, generator_kl_loss=1.842, generator_dur_loss=1.554, generator_adv_loss=1.793, generator_feat_match_loss=2.172, over 51.00 samples.], tot_loss[discriminator_loss=2.747, discriminator_real_loss=1.407, discriminator_fake_loss=1.34, generator_loss=30.06, generator_mel_loss=22.3, generator_kl_loss=1.91, generator_dur_loss=1.535, generator_adv_loss=1.891, generator_feat_match_loss=2.426, over 2380.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 128.0
2024-02-22 19:10:46,817 INFO [train.py:845] (2/4) Start epoch 162
2024-02-22 19:14:19,487 INFO [train.py:845] (2/4) Start epoch 163
2024-02-22 19:15:10,146 INFO [train.py:471] (2/4) Epoch 163, batch 6, global_batch_idx: 6000, batch size: 153, loss[discriminator_loss=2.812, discriminator_real_loss=1.534, discriminator_fake_loss=1.279, generator_loss=30.42, generator_mel_loss=22.39, generator_kl_loss=2.001, generator_dur_loss=1.527, generator_adv_loss=1.988, generator_feat_match_loss=2.51, over 153.00 samples.], tot_loss[discriminator_loss=2.874, discriminator_real_loss=1.497, discriminator_fake_loss=1.377, generator_loss=30.4, generator_mel_loss=22.48, generator_kl_loss=1.966, generator_dur_loss=1.532, generator_adv_loss=1.98, generator_feat_match_loss=2.439, over 613.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:15:10,147 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 19:15:18,966 INFO [train.py:534] (2/4) Epoch 163, validation: discriminator_loss=2.726, discriminator_real_loss=1.517, discriminator_fake_loss=1.209, generator_loss=31.62, generator_mel_loss=23.67, generator_kl_loss=2.024, generator_dur_loss=1.523, generator_adv_loss=1.958, generator_feat_match_loss=2.451, over 100.00 samples.
2024-02-22 19:15:18,967 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 19:17:57,870 INFO [train.py:845] (2/4) Start epoch 164
2024-02-22 19:19:53,627 INFO [train.py:471] (2/4) Epoch 164, batch 19, global_batch_idx: 6050, batch size: 51, loss[discriminator_loss=2.705, discriminator_real_loss=1.328, discriminator_fake_loss=1.377, generator_loss=30.72, generator_mel_loss=22.72, generator_kl_loss=1.92, generator_dur_loss=1.551, generator_adv_loss=2.016, generator_feat_match_loss=2.514, over 51.00 samples.], tot_loss[discriminator_loss=2.702, discriminator_real_loss=1.387, discriminator_fake_loss=1.315, generator_loss=30.33, generator_mel_loss=22.37, generator_kl_loss=1.935, generator_dur_loss=1.538, generator_adv_loss=1.926, generator_feat_match_loss=2.561, over 1381.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:21:27,729 INFO [train.py:845] (2/4) Start epoch 165
2024-02-22 19:24:28,178 INFO [train.py:471] (2/4) Epoch 165, batch 32, global_batch_idx: 6100, batch size: 71, loss[discriminator_loss=2.73, discriminator_real_loss=1.293, discriminator_fake_loss=1.438, generator_loss=30.1, generator_mel_loss=22.28, generator_kl_loss=1.899, generator_dur_loss=1.518, generator_adv_loss=1.987, generator_feat_match_loss=2.414, over 71.00 samples.], tot_loss[discriminator_loss=2.722, discriminator_real_loss=1.39, discriminator_fake_loss=1.332, generator_loss=30.13, generator_mel_loss=22.3, generator_kl_loss=1.898, generator_dur_loss=1.536, generator_adv_loss=1.909, generator_feat_match_loss=2.482, over 2349.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:24:48,843 INFO [train.py:845] (2/4) Start epoch 166
2024-02-22 19:28:22,561 INFO [train.py:845] (2/4) Start epoch 167
2024-02-22 19:29:23,716 INFO [train.py:471] (2/4) Epoch 167, batch 8, global_batch_idx: 6150, batch size: 85, loss[discriminator_loss=2.719, discriminator_real_loss=1.324, discriminator_fake_loss=1.395, generator_loss=30.74, generator_mel_loss=22.73, generator_kl_loss=1.931, generator_dur_loss=1.542, generator_adv_loss=2.023, generator_feat_match_loss=2.514, over 85.00 samples.], tot_loss[discriminator_loss=2.721, discriminator_real_loss=1.385, discriminator_fake_loss=1.337, generator_loss=30.11, generator_mel_loss=22.28, generator_kl_loss=1.887, generator_dur_loss=1.537, generator_adv_loss=1.93, generator_feat_match_loss=2.484, over 613.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:31:52,146 INFO [train.py:845] (2/4) Start epoch 168
2024-02-22 19:34:10,704 INFO [train.py:471] (2/4) Epoch 168, batch 21, global_batch_idx: 6200, batch size: 65, loss[discriminator_loss=2.734, discriminator_real_loss=1.332, discriminator_fake_loss=1.402, generator_loss=29.71, generator_mel_loss=21.62, generator_kl_loss=1.966, generator_dur_loss=1.562, generator_adv_loss=2.156, generator_feat_match_loss=2.406, over 65.00 samples.], tot_loss[discriminator_loss=2.705, discriminator_real_loss=1.355, discriminator_fake_loss=1.35, generator_loss=30.3, generator_mel_loss=22.32, generator_kl_loss=1.916, generator_dur_loss=1.534, generator_adv_loss=1.949, generator_feat_match_loss=2.577, over 1792.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:34:10,705 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 19:34:19,823 INFO [train.py:534] (2/4) Epoch 168, validation: discriminator_loss=2.789, discriminator_real_loss=1.408, discriminator_fake_loss=1.381, generator_loss=30.81, generator_mel_loss=22.79, generator_kl_loss=2.045, generator_dur_loss=1.523, generator_adv_loss=1.939, generator_feat_match_loss=2.519, over 100.00 samples.
2024-02-22 19:34:19,824 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 19:35:34,513 INFO [train.py:845] (2/4) Start epoch 169
2024-02-22 19:38:55,351 INFO [train.py:471] (2/4) Epoch 169, batch 34, global_batch_idx: 6250, batch size: 55, loss[discriminator_loss=2.699, discriminator_real_loss=1.309, discriminator_fake_loss=1.391, generator_loss=29.71, generator_mel_loss=21.96, generator_kl_loss=1.96, generator_dur_loss=1.572, generator_adv_loss=1.791, generator_feat_match_loss=2.426, over 55.00 samples.], tot_loss[discriminator_loss=2.721, discriminator_real_loss=1.387, discriminator_fake_loss=1.334, generator_loss=30.1, generator_mel_loss=22.15, generator_kl_loss=1.941, generator_dur_loss=1.535, generator_adv_loss=1.925, generator_feat_match_loss=2.544, over 2473.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:39:05,077 INFO [train.py:845] (2/4) Start epoch 170
2024-02-22 19:42:35,203 INFO [train.py:845] (2/4) Start epoch 171
2024-02-22 19:43:47,671 INFO [train.py:471] (2/4) Epoch 171, batch 10, global_batch_idx: 6300, batch size: 52, loss[discriminator_loss=2.766, discriminator_real_loss=1.623, discriminator_fake_loss=1.143, generator_loss=29.17, generator_mel_loss=21.74, generator_kl_loss=1.841, generator_dur_loss=1.545, generator_adv_loss=1.779, generator_feat_match_loss=2.27, over 52.00 samples.], tot_loss[discriminator_loss=2.712, discriminator_real_loss=1.4, discriminator_fake_loss=1.313, generator_loss=29.81, generator_mel_loss=22.01, generator_kl_loss=1.89, generator_dur_loss=1.537, generator_adv_loss=1.915, generator_feat_match_loss=2.457, over 675.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:46:12,348 INFO [train.py:845] (2/4) Start epoch 172
2024-02-22 19:48:22,595 INFO [train.py:471] (2/4) Epoch 172, batch 23, global_batch_idx: 6350, batch size: 59, loss[discriminator_loss=2.742, discriminator_real_loss=1.486, discriminator_fake_loss=1.257, generator_loss=30.15, generator_mel_loss=22.34, generator_kl_loss=1.924, generator_dur_loss=1.523, generator_adv_loss=1.854, generator_feat_match_loss=2.508, over 59.00 samples.], tot_loss[discriminator_loss=2.717, discriminator_real_loss=1.392, discriminator_fake_loss=1.326, generator_loss=29.98, generator_mel_loss=22.13, generator_kl_loss=1.899, generator_dur_loss=1.542, generator_adv_loss=1.895, generator_feat_match_loss=2.51, over 1496.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:49:38,494 INFO [train.py:845] (2/4) Start epoch 173
2024-02-22 19:53:07,605 INFO [train.py:471] (2/4) Epoch 173, batch 36, global_batch_idx: 6400, batch size: 51, loss[discriminator_loss=2.746, discriminator_real_loss=1.457, discriminator_fake_loss=1.288, generator_loss=29.75, generator_mel_loss=22.04, generator_kl_loss=1.922, generator_dur_loss=1.54, generator_adv_loss=1.748, generator_feat_match_loss=2.498, over 51.00 samples.], tot_loss[discriminator_loss=2.716, discriminator_real_loss=1.398, discriminator_fake_loss=1.318, generator_loss=29.96, generator_mel_loss=22.08, generator_kl_loss=1.901, generator_dur_loss=1.533, generator_adv_loss=1.909, generator_feat_match_loss=2.54, over 2611.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 19:53:07,606 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 19:53:15,945 INFO [train.py:534] (2/4) Epoch 173, validation: discriminator_loss=2.746, discriminator_real_loss=1.216, discriminator_fake_loss=1.53, generator_loss=30.82, generator_mel_loss=22.97, generator_kl_loss=2.056, generator_dur_loss=1.529, generator_adv_loss=1.662, generator_feat_match_loss=2.599, over 100.00 samples.
2024-02-22 19:53:15,945 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 19:53:16,412 INFO [train.py:845] (2/4) Start epoch 174
2024-02-22 19:56:49,241 INFO [train.py:845] (2/4) Start epoch 175
2024-02-22 19:58:14,361 INFO [train.py:471] (2/4) Epoch 175, batch 12, global_batch_idx: 6450, batch size: 58, loss[discriminator_loss=2.734, discriminator_real_loss=1.376, discriminator_fake_loss=1.357, generator_loss=29.66, generator_mel_loss=21.85, generator_kl_loss=1.868, generator_dur_loss=1.532, generator_adv_loss=1.986, generator_feat_match_loss=2.428, over 58.00 samples.], tot_loss[discriminator_loss=2.704, discriminator_real_loss=1.369, discriminator_fake_loss=1.335, generator_loss=30.05, generator_mel_loss=22.15, generator_kl_loss=1.925, generator_dur_loss=1.533, generator_adv_loss=1.9, generator_feat_match_loss=2.537, over 1087.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 20:00:21,909 INFO [train.py:845] (2/4) Start epoch 176
2024-02-22 20:02:44,737 INFO [train.py:471] (2/4) Epoch 176, batch 25, global_batch_idx: 6500, batch size: 110, loss[discriminator_loss=2.766, discriminator_real_loss=1.452, discriminator_fake_loss=1.313, generator_loss=30.85, generator_mel_loss=22.98, generator_kl_loss=1.969, generator_dur_loss=1.53, generator_adv_loss=1.768, generator_feat_match_loss=2.602, over 110.00 samples.], tot_loss[discriminator_loss=2.698, discriminator_real_loss=1.38, discriminator_fake_loss=1.318, generator_loss=30.18, generator_mel_loss=22.21, generator_kl_loss=1.924, generator_dur_loss=1.529, generator_adv_loss=1.933, generator_feat_match_loss=2.586, over 1800.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 20:03:49,138 INFO [train.py:845] (2/4) Start epoch 177
2024-02-22 20:07:16,910 INFO [train.py:845] (2/4) Start epoch 178
2024-02-22 20:07:34,522 INFO [train.py:471] (2/4) Epoch 178, batch 1, global_batch_idx: 6550, batch size: 52, loss[discriminator_loss=2.746, discriminator_real_loss=1.502, discriminator_fake_loss=1.244, generator_loss=30.56, generator_mel_loss=22.59, generator_kl_loss=1.956, generator_dur_loss=1.561, generator_adv_loss=1.855, generator_feat_match_loss=2.604, over 52.00 samples.], tot_loss[discriminator_loss=2.687, discriminator_real_loss=1.423, discriminator_fake_loss=1.264, generator_loss=30.5, generator_mel_loss=22.42, generator_kl_loss=1.899, generator_dur_loss=1.541, generator_adv_loss=1.979, generator_feat_match_loss=2.669, over 119.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 20:10:45,305 INFO [train.py:845] (2/4) Start epoch 179
2024-02-22 20:12:11,804 INFO [train.py:471] (2/4) Epoch 179, batch 14, global_batch_idx: 6600, batch size: 50, loss[discriminator_loss=2.609, discriminator_real_loss=1.254, discriminator_fake_loss=1.355, generator_loss=31.46, generator_mel_loss=23.03, generator_kl_loss=1.931, generator_dur_loss=1.545, generator_adv_loss=2.152, generator_feat_match_loss=2.805, over 50.00 samples.], tot_loss[discriminator_loss=2.727, discriminator_real_loss=1.387, discriminator_fake_loss=1.339, generator_loss=29.85, generator_mel_loss=22, generator_kl_loss=1.883, generator_dur_loss=1.532, generator_adv_loss=1.905, generator_feat_match_loss=2.528, over 1041.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 20:12:11,806 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 20:12:20,625 INFO [train.py:534] (2/4) Epoch 179, validation: discriminator_loss=2.647, discriminator_real_loss=1.388, discriminator_fake_loss=1.259, generator_loss=31.86, generator_mel_loss=23.53, generator_kl_loss=2.002, generator_dur_loss=1.527, generator_adv_loss=1.959, generator_feat_match_loss=2.849, over 100.00 samples.
2024-02-22 20:12:20,626 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28098MB
2024-02-22 20:14:19,164 INFO [train.py:845] (2/4) Start epoch 180
2024-02-22 20:17:04,393 INFO [train.py:471] (2/4) Epoch 180, batch 27, global_batch_idx: 6650, batch size: 76, loss[discriminator_loss=2.693, discriminator_real_loss=1.125, discriminator_fake_loss=1.568, generator_loss=30.8, generator_mel_loss=22.4, generator_kl_loss=1.796, generator_dur_loss=1.57, generator_adv_loss=2.443, generator_feat_match_loss=2.59, over 76.00 samples.], tot_loss[discriminator_loss=2.714, discriminator_real_loss=1.385, discriminator_fake_loss=1.329, generator_loss=30.2, generator_mel_loss=22.25, generator_kl_loss=1.916, generator_dur_loss=1.531, generator_adv_loss=1.943, generator_feat_match_loss=2.565, over 2075.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 20:17:50,009 INFO [train.py:845] (2/4) Start epoch 181
2024-02-22 20:21:16,739 INFO [train.py:845] (2/4) Start epoch 182
2024-02-22 20:21:44,768 INFO [train.py:471] (2/4) Epoch 182, batch 3, global_batch_idx: 6700, batch size: 65, loss[discriminator_loss=2.688, discriminator_real_loss=1.439, discriminator_fake_loss=1.248, generator_loss=29.81, generator_mel_loss=21.83, generator_kl_loss=1.892, generator_dur_loss=1.518, generator_adv_loss=1.852, generator_feat_match_loss=2.715, over 65.00 samples.], tot_loss[discriminator_loss=2.695, discriminator_real_loss=1.377, discriminator_fake_loss=1.319, generator_loss=30.83, generator_mel_loss=22.73, generator_kl_loss=1.955, generator_dur_loss=1.523, generator_adv_loss=1.921, generator_feat_match_loss=2.699, over 315.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 20:24:39,568 INFO [train.py:845] (2/4) Start epoch 183
2024-02-22 20:26:20,648 INFO [train.py:471] (2/4) Epoch 183, batch 16, global_batch_idx: 6750, batch size: 153, loss[discriminator_loss=2.738, discriminator_real_loss=1.196, discriminator_fake_loss=1.543, generator_loss=30.41, generator_mel_loss=22.04, generator_kl_loss=1.911, generator_dur_loss=1.507, generator_adv_loss=2.377, generator_feat_match_loss=2.572, over 153.00 samples.], tot_loss[discriminator_loss=2.714, discriminator_real_loss=1.363, discriminator_fake_loss=1.351, generator_loss=30.13, generator_mel_loss=22.1, generator_kl_loss=1.906, generator_dur_loss=1.527, generator_adv_loss=1.964, generator_feat_match_loss=2.637, over 1298.00 samples.], cur_lr_g: 1.96e-04, cur_lr_d: 1.96e-04, grad_scale: 256.0
2024-02-22 20:28:09,018 INFO [train.py:845] (2/4) Start epoch 184
2024-02-22 20:31:04,456 INFO [train.py:471] (2/4) Epoch 184, batch 29, global_batch_idx: 6800, batch size: 60, loss[discriminator_loss=2.633, discriminator_real_loss=1.4, discriminator_fake_loss=1.231, generator_loss=30.34, generator_mel_loss=22.18, generator_kl_loss=1.898, generator_dur_loss=1.535, generator_adv_loss=1.967, generator_feat_match_loss=2.766, over 60.00 samples.], tot_loss[discriminator_loss=2.716, discriminator_real_loss=1.38, discriminator_fake_loss=1.336, generator_loss=30.19, generator_mel_loss=22.21, generator_kl_loss=1.917, generator_dur_loss=1.525, generator_adv_loss=1.927, generator_feat_match_loss=2.604, over 2280.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 20:31:04,458 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 20:31:12,975 INFO [train.py:534] (2/4) Epoch 184, validation: discriminator_loss=2.673, discriminator_real_loss=1.282, discriminator_fake_loss=1.391, generator_loss=31.15, generator_mel_loss=23.06, generator_kl_loss=1.91, generator_dur_loss=1.511, generator_adv_loss=1.797, generator_feat_match_loss=2.872, over 100.00 samples.
2024-02-22 20:31:12,976 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 20:31:47,016 INFO [train.py:845] (2/4) Start epoch 185
2024-02-22 20:35:15,997 INFO [train.py:845] (2/4) Start epoch 186
2024-02-22 20:35:54,672 INFO [train.py:471] (2/4) Epoch 186, batch 5, global_batch_idx: 6850, batch size: 67, loss[discriminator_loss=2.695, discriminator_real_loss=1.539, discriminator_fake_loss=1.157, generator_loss=30.2, generator_mel_loss=22.26, generator_kl_loss=1.904, generator_dur_loss=1.528, generator_adv_loss=1.751, generator_feat_match_loss=2.75, over 67.00 samples.], tot_loss[discriminator_loss=2.724, discriminator_real_loss=1.384, discriminator_fake_loss=1.34, generator_loss=30.18, generator_mel_loss=22.21, generator_kl_loss=1.87, generator_dur_loss=1.519, generator_adv_loss=1.909, generator_feat_match_loss=2.67, over 493.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 20:38:40,502 INFO [train.py:845] (2/4) Start epoch 187
2024-02-22 20:40:33,506 INFO [train.py:471] (2/4) Epoch 187, batch 18, global_batch_idx: 6900, batch size: 65, loss[discriminator_loss=2.746, discriminator_real_loss=1.37, discriminator_fake_loss=1.376, generator_loss=29.68, generator_mel_loss=21.45, generator_kl_loss=2.031, generator_dur_loss=1.521, generator_adv_loss=2.242, generator_feat_match_loss=2.441, over 65.00 samples.], tot_loss[discriminator_loss=2.73, discriminator_real_loss=1.4, discriminator_fake_loss=1.33, generator_loss=30.03, generator_mel_loss=22.02, generator_kl_loss=1.902, generator_dur_loss=1.533, generator_adv_loss=1.959, generator_feat_match_loss=2.617, over 1272.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 20:42:12,409 INFO [train.py:845] (2/4) Start epoch 188
2024-02-22 20:45:05,577 INFO [train.py:471] (2/4) Epoch 188, batch 31, global_batch_idx: 6950, batch size: 95, loss[discriminator_loss=2.727, discriminator_real_loss=1.503, discriminator_fake_loss=1.225, generator_loss=29.99, generator_mel_loss=22.07, generator_kl_loss=1.947, generator_dur_loss=1.536, generator_adv_loss=1.882, generator_feat_match_loss=2.551, over 95.00 samples.], tot_loss[discriminator_loss=2.701, discriminator_real_loss=1.39, discriminator_fake_loss=1.311, generator_loss=30.09, generator_mel_loss=22.09, generator_kl_loss=1.91, generator_dur_loss=1.527, generator_adv_loss=1.934, generator_feat_match_loss=2.63, over 2438.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 20:45:37,082 INFO [train.py:845] (2/4) Start epoch 189
2024-02-22 20:49:06,522 INFO [train.py:845] (2/4) Start epoch 190
2024-02-22 20:50:06,275 INFO [train.py:471] (2/4) Epoch 190, batch 7, global_batch_idx: 7000, batch size: 153, loss[discriminator_loss=2.643, discriminator_real_loss=1.344, discriminator_fake_loss=1.299, generator_loss=30.03, generator_mel_loss=21.85, generator_kl_loss=1.955, generator_dur_loss=1.509, generator_adv_loss=1.924, generator_feat_match_loss=2.785, over 153.00 samples.], tot_loss[discriminator_loss=2.683, discriminator_real_loss=1.365, discriminator_fake_loss=1.318, generator_loss=30.42, generator_mel_loss=22.29, generator_kl_loss=1.957, generator_dur_loss=1.519, generator_adv_loss=1.936, generator_feat_match_loss=2.716, over 779.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 20:50:06,277 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 20:50:14,851 INFO [train.py:534] (2/4) Epoch 190, validation: discriminator_loss=2.657, discriminator_real_loss=1.299, discriminator_fake_loss=1.358, generator_loss=31.16, generator_mel_loss=23, generator_kl_loss=2.042, generator_dur_loss=1.524, generator_adv_loss=1.778, generator_feat_match_loss=2.815, over 100.00 samples.
2024-02-22 20:50:14,852 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 20:52:48,749 INFO [train.py:845] (2/4) Start epoch 191
2024-02-22 20:54:46,328 INFO [train.py:471] (2/4) Epoch 191, batch 20, global_batch_idx: 7050, batch size: 95, loss[discriminator_loss=2.67, discriminator_real_loss=1.326, discriminator_fake_loss=1.344, generator_loss=29.62, generator_mel_loss=21.77, generator_kl_loss=1.855, generator_dur_loss=1.52, generator_adv_loss=1.827, generator_feat_match_loss=2.65, over 95.00 samples.], tot_loss[discriminator_loss=2.702, discriminator_real_loss=1.379, discriminator_fake_loss=1.323, generator_loss=30.01, generator_mel_loss=22.01, generator_kl_loss=1.891, generator_dur_loss=1.53, generator_adv_loss=1.927, generator_feat_match_loss=2.646, over 1573.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 20:56:14,525 INFO [train.py:845] (2/4) Start epoch 192
2024-02-22 20:59:26,503 INFO [train.py:471] (2/4) Epoch 192, batch 33, global_batch_idx: 7100, batch size: 60, loss[discriminator_loss=2.693, discriminator_real_loss=1.366, discriminator_fake_loss=1.327, generator_loss=29.98, generator_mel_loss=22.22, generator_kl_loss=1.793, generator_dur_loss=1.541, generator_adv_loss=1.819, generator_feat_match_loss=2.605, over 60.00 samples.], tot_loss[discriminator_loss=2.725, discriminator_real_loss=1.389, discriminator_fake_loss=1.337, generator_loss=29.87, generator_mel_loss=21.91, generator_kl_loss=1.911, generator_dur_loss=1.523, generator_adv_loss=1.936, generator_feat_match_loss=2.597, over 2432.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 20:59:43,001 INFO [train.py:845] (2/4) Start epoch 193
2024-02-22 21:03:02,593 INFO [train.py:845] (2/4) Start epoch 194
2024-02-22 21:03:57,678 INFO [train.py:471] (2/4) Epoch 194, batch 9, global_batch_idx: 7150, batch size: 81, loss[discriminator_loss=2.652, discriminator_real_loss=1.555, discriminator_fake_loss=1.097, generator_loss=29.77, generator_mel_loss=21.89, generator_kl_loss=1.854, generator_dur_loss=1.511, generator_adv_loss=1.733, generator_feat_match_loss=2.777, over 81.00 samples.], tot_loss[discriminator_loss=2.689, discriminator_real_loss=1.373, discriminator_fake_loss=1.316, generator_loss=29.76, generator_mel_loss=21.83, generator_kl_loss=1.892, generator_dur_loss=1.529, generator_adv_loss=1.903, generator_feat_match_loss=2.61, over 694.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:06:24,892 INFO [train.py:845] (2/4) Start epoch 195
2024-02-22 21:08:23,956 INFO [train.py:471] (2/4) Epoch 195, batch 22, global_batch_idx: 7200, batch size: 73, loss[discriminator_loss=2.707, discriminator_real_loss=1.482, discriminator_fake_loss=1.226, generator_loss=30.04, generator_mel_loss=21.86, generator_kl_loss=1.989, generator_dur_loss=1.521, generator_adv_loss=1.919, generator_feat_match_loss=2.752, over 73.00 samples.], tot_loss[discriminator_loss=2.727, discriminator_real_loss=1.391, discriminator_fake_loss=1.337, generator_loss=30.1, generator_mel_loss=22.1, generator_kl_loss=1.895, generator_dur_loss=1.531, generator_adv_loss=1.932, generator_feat_match_loss=2.641, over 1572.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:08:23,957 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 21:08:31,775 INFO [train.py:534] (2/4) Epoch 195, validation: discriminator_loss=2.647, discriminator_real_loss=1.321, discriminator_fake_loss=1.326, generator_loss=31.38, generator_mel_loss=23.32, generator_kl_loss=1.954, generator_dur_loss=1.517, generator_adv_loss=1.801, generator_feat_match_loss=2.783, over 100.00 samples.
2024-02-22 21:08:31,776 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 21:09:54,101 INFO [train.py:845] (2/4) Start epoch 196
2024-02-22 21:13:15,888 INFO [train.py:471] (2/4) Epoch 196, batch 35, global_batch_idx: 7250, batch size: 95, loss[discriminator_loss=2.773, discriminator_real_loss=1.282, discriminator_fake_loss=1.49, generator_loss=30.51, generator_mel_loss=22.43, generator_kl_loss=1.889, generator_dur_loss=1.504, generator_adv_loss=1.916, generator_feat_match_loss=2.773, over 95.00 samples.], tot_loss[discriminator_loss=2.714, discriminator_real_loss=1.371, discriminator_fake_loss=1.342, generator_loss=30.13, generator_mel_loss=22.12, generator_kl_loss=1.907, generator_dur_loss=1.525, generator_adv_loss=1.924, generator_feat_match_loss=2.652, over 2652.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:13:20,871 INFO [train.py:845] (2/4) Start epoch 197
2024-02-22 21:16:49,140 INFO [train.py:845] (2/4) Start epoch 198
2024-02-22 21:18:02,231 INFO [train.py:471] (2/4) Epoch 198, batch 11, global_batch_idx: 7300, batch size: 85, loss[discriminator_loss=2.742, discriminator_real_loss=1.285, discriminator_fake_loss=1.457, generator_loss=29.85, generator_mel_loss=21.86, generator_kl_loss=1.876, generator_dur_loss=1.513, generator_adv_loss=2.033, generator_feat_match_loss=2.566, over 85.00 samples.], tot_loss[discriminator_loss=2.715, discriminator_real_loss=1.381, discriminator_fake_loss=1.333, generator_loss=30.01, generator_mel_loss=22, generator_kl_loss=1.888, generator_dur_loss=1.532, generator_adv_loss=1.935, generator_feat_match_loss=2.654, over 812.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:20:14,196 INFO [train.py:845] (2/4) Start epoch 199
2024-02-22 21:22:39,862 INFO [train.py:471] (2/4) Epoch 199, batch 24, global_batch_idx: 7350, batch size: 56, loss[discriminator_loss=2.797, discriminator_real_loss=1.276, discriminator_fake_loss=1.52, generator_loss=30.71, generator_mel_loss=22.55, generator_kl_loss=1.858, generator_dur_loss=1.545, generator_adv_loss=1.922, generator_feat_match_loss=2.838, over 56.00 samples.], tot_loss[discriminator_loss=2.723, discriminator_real_loss=1.383, discriminator_fake_loss=1.339, generator_loss=30.06, generator_mel_loss=22.05, generator_kl_loss=1.891, generator_dur_loss=1.527, generator_adv_loss=1.928, generator_feat_match_loss=2.658, over 1782.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:23:45,334 INFO [train.py:845] (2/4) Start epoch 200
2024-02-22 21:27:14,714 INFO [train.py:845] (2/4) Start epoch 201
2024-02-22 21:27:29,793 INFO [train.py:471] (2/4) Epoch 201, batch 0, global_batch_idx: 7400, batch size: 85, loss[discriminator_loss=2.766, discriminator_real_loss=1.373, discriminator_fake_loss=1.392, generator_loss=29.85, generator_mel_loss=21.77, generator_kl_loss=2.039, generator_dur_loss=1.503, generator_adv_loss=1.928, generator_feat_match_loss=2.613, over 85.00 samples.], tot_loss[discriminator_loss=2.766, discriminator_real_loss=1.373, discriminator_fake_loss=1.392, generator_loss=29.85, generator_mel_loss=21.77, generator_kl_loss=2.039, generator_dur_loss=1.503, generator_adv_loss=1.928, generator_feat_match_loss=2.613, over 85.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:27:29,794 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 21:27:38,807 INFO [train.py:534] (2/4) Epoch 201, validation: discriminator_loss=2.61, discriminator_real_loss=1.297, discriminator_fake_loss=1.313, generator_loss=30.93, generator_mel_loss=22.73, generator_kl_loss=2.003, generator_dur_loss=1.521, generator_adv_loss=1.826, generator_feat_match_loss=2.848, over 100.00 samples.
2024-02-22 21:27:38,808 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 21:30:58,177 INFO [train.py:845] (2/4) Start epoch 202
2024-02-22 21:32:23,989 INFO [train.py:471] (2/4) Epoch 202, batch 13, global_batch_idx: 7450, batch size: 60, loss[discriminator_loss=2.785, discriminator_real_loss=1.479, discriminator_fake_loss=1.308, generator_loss=30.14, generator_mel_loss=22.23, generator_kl_loss=1.966, generator_dur_loss=1.511, generator_adv_loss=1.938, generator_feat_match_loss=2.49, over 60.00 samples.], tot_loss[discriminator_loss=2.707, discriminator_real_loss=1.361, discriminator_fake_loss=1.345, generator_loss=30.23, generator_mel_loss=22.14, generator_kl_loss=1.936, generator_dur_loss=1.514, generator_adv_loss=1.923, generator_feat_match_loss=2.72, over 1115.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:34:29,362 INFO [train.py:845] (2/4) Start epoch 203
2024-02-22 21:36:57,120 INFO [train.py:471] (2/4) Epoch 203, batch 26, global_batch_idx: 7500, batch size: 54, loss[discriminator_loss=2.719, discriminator_real_loss=1.519, discriminator_fake_loss=1.2, generator_loss=29.08, generator_mel_loss=21.38, generator_kl_loss=2.005, generator_dur_loss=1.552, generator_adv_loss=1.807, generator_feat_match_loss=2.334, over 54.00 samples.], tot_loss[discriminator_loss=2.751, discriminator_real_loss=1.4, discriminator_fake_loss=1.351, generator_loss=29.95, generator_mel_loss=21.97, generator_kl_loss=1.908, generator_dur_loss=1.523, generator_adv_loss=1.931, generator_feat_match_loss=2.617, over 1874.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:37:57,100 INFO [train.py:845] (2/4) Start epoch 204
2024-02-22 21:41:31,597 INFO [train.py:845] (2/4) Start epoch 205
2024-02-22 21:41:55,778 INFO [train.py:471] (2/4) Epoch 205, batch 2, global_batch_idx: 7550, batch size: 73, loss[discriminator_loss=2.695, discriminator_real_loss=1.482, discriminator_fake_loss=1.212, generator_loss=29.88, generator_mel_loss=21.86, generator_kl_loss=1.951, generator_dur_loss=1.535, generator_adv_loss=1.919, generator_feat_match_loss=2.617, over 73.00 samples.], tot_loss[discriminator_loss=2.703, discriminator_real_loss=1.357, discriminator_fake_loss=1.345, generator_loss=29.91, generator_mel_loss=21.94, generator_kl_loss=1.921, generator_dur_loss=1.524, generator_adv_loss=1.914, generator_feat_match_loss=2.607, over 278.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:44:54,677 INFO [train.py:845] (2/4) Start epoch 206
2024-02-22 21:46:27,674 INFO [train.py:471] (2/4) Epoch 206, batch 15, global_batch_idx: 7600, batch size: 101, loss[discriminator_loss=2.75, discriminator_real_loss=1.559, discriminator_fake_loss=1.192, generator_loss=29.28, generator_mel_loss=21.51, generator_kl_loss=1.948, generator_dur_loss=1.502, generator_adv_loss=1.688, generator_feat_match_loss=2.631, over 101.00 samples.], tot_loss[discriminator_loss=2.738, discriminator_real_loss=1.412, discriminator_fake_loss=1.326, generator_loss=29.98, generator_mel_loss=21.94, generator_kl_loss=1.915, generator_dur_loss=1.526, generator_adv_loss=1.918, generator_feat_match_loss=2.681, over 1226.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 256.0
2024-02-22 21:46:27,676 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 21:46:36,589 INFO [train.py:534] (2/4) Epoch 206, validation: discriminator_loss=2.762, discriminator_real_loss=1.162, discriminator_fake_loss=1.6, generator_loss=30.51, generator_mel_loss=22.59, generator_kl_loss=1.973, generator_dur_loss=1.512, generator_adv_loss=1.587, generator_feat_match_loss=2.845, over 100.00 samples.
2024-02-22 21:46:36,590 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 21:48:34,003 INFO [train.py:845] (2/4) Start epoch 207
2024-02-22 21:51:21,107 INFO [train.py:471] (2/4) Epoch 207, batch 28, global_batch_idx: 7650, batch size: 59, loss[discriminator_loss=2.693, discriminator_real_loss=1.387, discriminator_fake_loss=1.307, generator_loss=30.07, generator_mel_loss=21.94, generator_kl_loss=1.778, generator_dur_loss=1.527, generator_adv_loss=1.982, generator_feat_match_loss=2.838, over 59.00 samples.], tot_loss[discriminator_loss=2.72, discriminator_real_loss=1.379, discriminator_fake_loss=1.342, generator_loss=30.01, generator_mel_loss=21.89, generator_kl_loss=1.9, generator_dur_loss=1.517, generator_adv_loss=1.962, generator_feat_match_loss=2.741, over 2241.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 21:52:02,716 INFO [train.py:845] (2/4) Start epoch 208
2024-02-22 21:55:28,006 INFO [train.py:845] (2/4) Start epoch 209
2024-02-22 21:56:00,310 INFO [train.py:471] (2/4) Epoch 209, batch 4, global_batch_idx: 7700, batch size: 58, loss[discriminator_loss=2.723, discriminator_real_loss=1.167, discriminator_fake_loss=1.556, generator_loss=30.17, generator_mel_loss=21.75, generator_kl_loss=1.909, generator_dur_loss=1.515, generator_adv_loss=2.285, generator_feat_match_loss=2.715, over 58.00 samples.], tot_loss[discriminator_loss=2.688, discriminator_real_loss=1.379, discriminator_fake_loss=1.309, generator_loss=29.6, generator_mel_loss=21.52, generator_kl_loss=1.872, generator_dur_loss=1.534, generator_adv_loss=1.961, generator_feat_match_loss=2.711, over 302.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 21:58:57,925 INFO [train.py:845] (2/4) Start epoch 210
2024-02-22 22:00:47,521 INFO [train.py:471] (2/4) Epoch 210, batch 17, global_batch_idx: 7750, batch size: 56, loss[discriminator_loss=2.707, discriminator_real_loss=1.46, discriminator_fake_loss=1.247, generator_loss=29.73, generator_mel_loss=21.82, generator_kl_loss=1.863, generator_dur_loss=1.541, generator_adv_loss=1.885, generator_feat_match_loss=2.625, over 56.00 samples.], tot_loss[discriminator_loss=2.71, discriminator_real_loss=1.384, discriminator_fake_loss=1.326, generator_loss=30.02, generator_mel_loss=21.94, generator_kl_loss=1.922, generator_dur_loss=1.523, generator_adv_loss=1.928, generator_feat_match_loss=2.713, over 1486.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 22:02:33,485 INFO [train.py:845] (2/4) Start epoch 211
2024-02-22 22:05:39,916 INFO [train.py:471] (2/4) Epoch 211, batch 30, global_batch_idx: 7800, batch size: 126, loss[discriminator_loss=2.715, discriminator_real_loss=1.31, discriminator_fake_loss=1.404, generator_loss=30.74, generator_mel_loss=22.44, generator_kl_loss=1.91, generator_dur_loss=1.517, generator_adv_loss=2.057, generator_feat_match_loss=2.826, over 126.00 samples.], tot_loss[discriminator_loss=2.73, discriminator_real_loss=1.379, discriminator_fake_loss=1.351, generator_loss=30.07, generator_mel_loss=22, generator_kl_loss=1.915, generator_dur_loss=1.522, generator_adv_loss=1.94, generator_feat_match_loss=2.692, over 2284.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 22:05:39,918 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 22:05:48,285 INFO [train.py:534] (2/4) Epoch 211, validation: discriminator_loss=2.691, discriminator_real_loss=1.395, discriminator_fake_loss=1.297, generator_loss=30.81, generator_mel_loss=22.59, generator_kl_loss=1.987, generator_dur_loss=1.519, generator_adv_loss=1.979, generator_feat_match_loss=2.739, over 100.00 samples.
2024-02-22 22:05:48,286 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 22:06:15,414 INFO [train.py:845] (2/4) Start epoch 212
2024-02-22 22:09:34,290 INFO [train.py:845] (2/4) Start epoch 213
2024-02-22 22:10:23,323 INFO [train.py:471] (2/4) Epoch 213, batch 6, global_batch_idx: 7850, batch size: 110, loss[discriminator_loss=2.658, discriminator_real_loss=1.439, discriminator_fake_loss=1.219, generator_loss=30.66, generator_mel_loss=22.46, generator_kl_loss=1.985, generator_dur_loss=1.515, generator_adv_loss=1.885, generator_feat_match_loss=2.811, over 110.00 samples.], tot_loss[discriminator_loss=2.673, discriminator_real_loss=1.362, discriminator_fake_loss=1.31, generator_loss=30.21, generator_mel_loss=22.06, generator_kl_loss=1.953, generator_dur_loss=1.509, generator_adv_loss=1.926, generator_feat_match_loss=2.767, over 619.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 22:12:59,221 INFO [train.py:845] (2/4) Start epoch 214
2024-02-22 22:14:52,761 INFO [train.py:471] (2/4) Epoch 214, batch 19, global_batch_idx: 7900, batch size: 56, loss[discriminator_loss=2.754, discriminator_real_loss=1.391, discriminator_fake_loss=1.364, generator_loss=29.98, generator_mel_loss=21.89, generator_kl_loss=1.942, generator_dur_loss=1.542, generator_adv_loss=2.021, generator_feat_match_loss=2.59, over 56.00 samples.], tot_loss[discriminator_loss=2.71, discriminator_real_loss=1.371, discriminator_fake_loss=1.339, generator_loss=29.88, generator_mel_loss=21.82, generator_kl_loss=1.914, generator_dur_loss=1.525, generator_adv_loss=1.919, generator_feat_match_loss=2.698, over 1474.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 22:16:30,098 INFO [train.py:845] (2/4) Start epoch 215
2024-02-22 22:19:36,037 INFO [train.py:471] (2/4) Epoch 215, batch 32, global_batch_idx: 7950, batch size: 52, loss[discriminator_loss=2.73, discriminator_real_loss=1.279, discriminator_fake_loss=1.451, generator_loss=30.88, generator_mel_loss=22.29, generator_kl_loss=1.847, generator_dur_loss=1.545, generator_adv_loss=2.334, generator_feat_match_loss=2.863, over 52.00 samples.], tot_loss[discriminator_loss=2.725, discriminator_real_loss=1.373, discriminator_fake_loss=1.352, generator_loss=29.9, generator_mel_loss=21.85, generator_kl_loss=1.914, generator_dur_loss=1.525, generator_adv_loss=1.922, generator_feat_match_loss=2.689, over 2129.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 22:19:59,414 INFO [train.py:845] (2/4) Start epoch 216
2024-02-22 22:23:30,101 INFO [train.py:845] (2/4) Start epoch 217
2024-02-22 22:24:32,891 INFO [train.py:471] (2/4) Epoch 217, batch 8, global_batch_idx: 8000, batch size: 153, loss[discriminator_loss=2.709, discriminator_real_loss=1.426, discriminator_fake_loss=1.283, generator_loss=30.23, generator_mel_loss=22.09, generator_kl_loss=1.949, generator_dur_loss=1.518, generator_adv_loss=1.858, generator_feat_match_loss=2.811, over 153.00 samples.], tot_loss[discriminator_loss=2.706, discriminator_real_loss=1.389, discriminator_fake_loss=1.317, generator_loss=30.05, generator_mel_loss=21.97, generator_kl_loss=1.899, generator_dur_loss=1.521, generator_adv_loss=1.921, generator_feat_match_loss=2.742, over 740.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 128.0
2024-02-22 22:24:32,893 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 22:24:40,501 INFO [train.py:534] (2/4) Epoch 217, validation: discriminator_loss=2.635, discriminator_real_loss=1.275, discriminator_fake_loss=1.36, generator_loss=30.71, generator_mel_loss=22.59, generator_kl_loss=1.966, generator_dur_loss=1.511, generator_adv_loss=1.781, generator_feat_match_loss=2.86, over 100.00 samples.
2024-02-22 22:24:40,501 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 22:27:06,502 INFO [train.py:845] (2/4) Start epoch 218
2024-02-22 22:29:07,962 INFO [train.py:471] (2/4) Epoch 218, batch 21, global_batch_idx: 8050, batch size: 126, loss[discriminator_loss=2.689, discriminator_real_loss=1.394, discriminator_fake_loss=1.296, generator_loss=30.27, generator_mel_loss=21.96, generator_kl_loss=1.978, generator_dur_loss=1.505, generator_adv_loss=2.041, generator_feat_match_loss=2.789, over 126.00 samples.], tot_loss[discriminator_loss=2.72, discriminator_real_loss=1.392, discriminator_fake_loss=1.328, generator_loss=29.9, generator_mel_loss=21.85, generator_kl_loss=1.908, generator_dur_loss=1.519, generator_adv_loss=1.93, generator_feat_match_loss=2.691, over 1661.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 64.0
2024-02-22 22:30:26,901 INFO [train.py:845] (2/4) Start epoch 219
2024-02-22 22:33:44,191 INFO [train.py:471] (2/4) Epoch 219, batch 34, global_batch_idx: 8100, batch size: 50, loss[discriminator_loss=2.57, discriminator_real_loss=1.223, discriminator_fake_loss=1.347, generator_loss=30.61, generator_mel_loss=22.16, generator_kl_loss=1.895, generator_dur_loss=1.518, generator_adv_loss=1.841, generator_feat_match_loss=3.203, over 50.00 samples.], tot_loss[discriminator_loss=2.712, discriminator_real_loss=1.376, discriminator_fake_loss=1.337, generator_loss=30.13, generator_mel_loss=21.9, generator_kl_loss=1.9, generator_dur_loss=1.51, generator_adv_loss=1.99, generator_feat_match_loss=2.826, over 2827.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 32.0
2024-02-22 22:33:52,700 INFO [train.py:845] (2/4) Start epoch 220
2024-02-22 22:37:18,100 INFO [train.py:845] (2/4) Start epoch 221
2024-02-22 22:38:29,879 INFO [train.py:471] (2/4) Epoch 221, batch 10, global_batch_idx: 8150, batch size: 153, loss[discriminator_loss=3.205, discriminator_real_loss=1.381, discriminator_fake_loss=1.824, generator_loss=31.52, generator_mel_loss=22.48, generator_kl_loss=1.944, generator_dur_loss=1.509, generator_adv_loss=2.85, generator_feat_match_loss=2.734, over 153.00 samples.], tot_loss[discriminator_loss=2.86, discriminator_real_loss=1.361, discriminator_fake_loss=1.499, generator_loss=31.15, generator_mel_loss=22.25, generator_kl_loss=1.925, generator_dur_loss=1.521, generator_adv_loss=2.358, generator_feat_match_loss=3.091, over 909.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 16.0
2024-02-22 22:40:45,397 INFO [train.py:845] (2/4) Start epoch 222
2024-02-22 22:43:06,514 INFO [train.py:471] (2/4) Epoch 222, batch 23, global_batch_idx: 8200, batch size: 71, loss[discriminator_loss=2.715, discriminator_real_loss=1.325, discriminator_fake_loss=1.391, generator_loss=30.69, generator_mel_loss=21.77, generator_kl_loss=1.782, generator_dur_loss=1.522, generator_adv_loss=2.334, generator_feat_match_loss=3.281, over 71.00 samples.], tot_loss[discriminator_loss=2.669, discriminator_real_loss=1.343, discriminator_fake_loss=1.326, generator_loss=30.01, generator_mel_loss=21.68, generator_kl_loss=1.895, generator_dur_loss=1.522, generator_adv_loss=2.034, generator_feat_match_loss=2.879, over 1878.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 16.0
2024-02-22 22:43:06,516 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 22:43:15,097 INFO [train.py:534] (2/4) Epoch 222, validation: discriminator_loss=2.618, discriminator_real_loss=1.228, discriminator_fake_loss=1.39, generator_loss=31.04, generator_mel_loss=22.83, generator_kl_loss=2.01, generator_dur_loss=1.511, generator_adv_loss=1.779, generator_feat_match_loss=2.91, over 100.00 samples.
2024-02-22 22:43:15,098 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 22:44:28,044 INFO [train.py:845] (2/4) Start epoch 223
2024-02-22 22:47:57,591 INFO [train.py:471] (2/4) Epoch 223, batch 36, global_batch_idx: 8250, batch size: 54, loss[discriminator_loss=2.758, discriminator_real_loss=1.391, discriminator_fake_loss=1.368, generator_loss=29.36, generator_mel_loss=21.16, generator_kl_loss=1.868, generator_dur_loss=1.511, generator_adv_loss=2.059, generator_feat_match_loss=2.766, over 54.00 samples.], tot_loss[discriminator_loss=2.694, discriminator_real_loss=1.351, discriminator_fake_loss=1.343, generator_loss=30.27, generator_mel_loss=21.87, generator_kl_loss=1.916, generator_dur_loss=1.518, generator_adv_loss=2.069, generator_feat_match_loss=2.902, over 2730.00 samples.], cur_lr_g: 1.95e-04, cur_lr_d: 1.95e-04, grad_scale: 16.0
2024-02-22 22:47:58,076 INFO [train.py:845] (2/4) Start epoch 224
2024-02-22 22:51:32,064 INFO [train.py:845] (2/4) Start epoch 225
2024-02-22 22:52:48,996 INFO [train.py:471] (2/4) Epoch 225, batch 12, global_batch_idx: 8300, batch size: 126, loss[discriminator_loss=2.756, discriminator_real_loss=1.537, discriminator_fake_loss=1.219, generator_loss=30.62, generator_mel_loss=22.28, generator_kl_loss=1.916, generator_dur_loss=1.525, generator_adv_loss=1.979, generator_feat_match_loss=2.92, over 126.00 samples.], tot_loss[discriminator_loss=2.654, discriminator_real_loss=1.349, discriminator_fake_loss=1.305, generator_loss=30.3, generator_mel_loss=21.85, generator_kl_loss=1.897, generator_dur_loss=1.517, generator_adv_loss=2.071, generator_feat_match_loss=2.965, over 1078.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 22:55:00,826 INFO [train.py:845] (2/4) Start epoch 226
2024-02-22 22:57:32,480 INFO [train.py:471] (2/4) Epoch 226, batch 25, global_batch_idx: 8350, batch size: 49, loss[discriminator_loss=2.689, discriminator_real_loss=1.361, discriminator_fake_loss=1.328, generator_loss=30.7, generator_mel_loss=21.85, generator_kl_loss=1.879, generator_dur_loss=1.54, generator_adv_loss=2.043, generator_feat_match_loss=3.389, over 49.00 samples.], tot_loss[discriminator_loss=2.734, discriminator_real_loss=1.424, discriminator_fake_loss=1.31, generator_loss=30.37, generator_mel_loss=21.67, generator_kl_loss=1.888, generator_dur_loss=1.517, generator_adv_loss=2.148, generator_feat_match_loss=3.142, over 1751.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 22:58:32,111 INFO [train.py:845] (2/4) Start epoch 227
2024-02-22 23:02:04,962 INFO [train.py:845] (2/4) Start epoch 228
2024-02-22 23:02:23,896 INFO [train.py:471] (2/4) Epoch 228, batch 1, global_batch_idx: 8400, batch size: 65, loss[discriminator_loss=2.697, discriminator_real_loss=1.307, discriminator_fake_loss=1.391, generator_loss=30.55, generator_mel_loss=21.72, generator_kl_loss=1.972, generator_dur_loss=1.531, generator_adv_loss=2.133, generator_feat_match_loss=3.195, over 65.00 samples.], tot_loss[discriminator_loss=2.625, discriminator_real_loss=1.3, discriminator_fake_loss=1.326, generator_loss=31.01, generator_mel_loss=21.83, generator_kl_loss=1.954, generator_dur_loss=1.529, generator_adv_loss=2.249, generator_feat_match_loss=3.451, over 117.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 32.0
2024-02-22 23:02:23,897 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 23:02:32,679 INFO [train.py:534] (2/4) Epoch 228, validation: discriminator_loss=2.758, discriminator_real_loss=1.256, discriminator_fake_loss=1.502, generator_loss=30.72, generator_mel_loss=22.26, generator_kl_loss=2.084, generator_dur_loss=1.504, generator_adv_loss=1.804, generator_feat_match_loss=3.063, over 100.00 samples.
2024-02-22 23:02:32,679 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 23:05:44,895 INFO [train.py:845] (2/4) Start epoch 229
2024-02-22 23:07:01,726 INFO [train.py:471] (2/4) Epoch 229, batch 14, global_batch_idx: 8450, batch size: 59, loss[discriminator_loss=2.574, discriminator_real_loss=1.357, discriminator_fake_loss=1.217, generator_loss=30.37, generator_mel_loss=21.57, generator_kl_loss=1.92, generator_dur_loss=1.522, generator_adv_loss=2.178, generator_feat_match_loss=3.18, over 59.00 samples.], tot_loss[discriminator_loss=2.631, discriminator_real_loss=1.315, discriminator_fake_loss=1.315, generator_loss=29.83, generator_mel_loss=21.54, generator_kl_loss=1.91, generator_dur_loss=1.527, generator_adv_loss=2.009, generator_feat_match_loss=2.85, over 940.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:09:08,700 INFO [train.py:845] (2/4) Start epoch 230
2024-02-22 23:11:47,829 INFO [train.py:471] (2/4) Epoch 230, batch 27, global_batch_idx: 8500, batch size: 60, loss[discriminator_loss=2.848, discriminator_real_loss=1.275, discriminator_fake_loss=1.571, generator_loss=29.39, generator_mel_loss=21.63, generator_kl_loss=1.956, generator_dur_loss=1.531, generator_adv_loss=1.91, generator_feat_match_loss=2.363, over 60.00 samples.], tot_loss[discriminator_loss=2.633, discriminator_real_loss=1.318, discriminator_fake_loss=1.315, generator_loss=30.34, generator_mel_loss=21.51, generator_kl_loss=1.898, generator_dur_loss=1.519, generator_adv_loss=2.185, generator_feat_match_loss=3.223, over 1954.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:12:39,936 INFO [train.py:845] (2/4) Start epoch 231
2024-02-22 23:16:09,065 INFO [train.py:845] (2/4) Start epoch 232
2024-02-22 23:16:38,292 INFO [train.py:471] (2/4) Epoch 232, batch 3, global_batch_idx: 8550, batch size: 53, loss[discriminator_loss=2.623, discriminator_real_loss=1.371, discriminator_fake_loss=1.252, generator_loss=30.42, generator_mel_loss=21.38, generator_kl_loss=1.882, generator_dur_loss=1.547, generator_adv_loss=2.336, generator_feat_match_loss=3.273, over 53.00 samples.], tot_loss[discriminator_loss=2.546, discriminator_real_loss=1.285, discriminator_fake_loss=1.262, generator_loss=30.19, generator_mel_loss=21.27, generator_kl_loss=1.877, generator_dur_loss=1.522, generator_adv_loss=2.268, generator_feat_match_loss=3.259, over 250.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:19:41,388 INFO [train.py:845] (2/4) Start epoch 233
2024-02-22 23:21:25,319 INFO [train.py:471] (2/4) Epoch 233, batch 16, global_batch_idx: 8600, batch size: 95, loss[discriminator_loss=2.449, discriminator_real_loss=1.285, discriminator_fake_loss=1.163, generator_loss=30.87, generator_mel_loss=21.43, generator_kl_loss=1.755, generator_dur_loss=1.526, generator_adv_loss=2.373, generator_feat_match_loss=3.779, over 95.00 samples.], tot_loss[discriminator_loss=2.529, discriminator_real_loss=1.286, discriminator_fake_loss=1.243, generator_loss=30.77, generator_mel_loss=21.45, generator_kl_loss=1.876, generator_dur_loss=1.519, generator_adv_loss=2.307, generator_feat_match_loss=3.619, over 1290.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:21:25,320 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 23:21:33,519 INFO [train.py:534] (2/4) Epoch 233, validation: discriminator_loss=2.418, discriminator_real_loss=1.299, discriminator_fake_loss=1.119, generator_loss=32.3, generator_mel_loss=22.62, generator_kl_loss=1.961, generator_dur_loss=1.516, generator_adv_loss=2.35, generator_feat_match_loss=3.86, over 100.00 samples.
2024-02-22 23:21:33,520 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 23:23:14,663 INFO [train.py:845] (2/4) Start epoch 234
2024-02-22 23:26:07,397 INFO [train.py:471] (2/4) Epoch 234, batch 29, global_batch_idx: 8650, batch size: 61, loss[discriminator_loss=2.594, discriminator_real_loss=1.097, discriminator_fake_loss=1.497, generator_loss=31.03, generator_mel_loss=21.57, generator_kl_loss=1.94, generator_dur_loss=1.509, generator_adv_loss=2.387, generator_feat_match_loss=3.617, over 61.00 samples.], tot_loss[discriminator_loss=2.571, discriminator_real_loss=1.319, discriminator_fake_loss=1.252, generator_loss=30.74, generator_mel_loss=21.45, generator_kl_loss=1.903, generator_dur_loss=1.514, generator_adv_loss=2.31, generator_feat_match_loss=3.561, over 2266.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:26:45,442 INFO [train.py:845] (2/4) Start epoch 235
2024-02-22 23:30:11,244 INFO [train.py:845] (2/4) Start epoch 236
2024-02-22 23:30:54,451 INFO [train.py:471] (2/4) Epoch 236, batch 5, global_batch_idx: 8700, batch size: 53, loss[discriminator_loss=2.484, discriminator_real_loss=1.218, discriminator_fake_loss=1.267, generator_loss=30.81, generator_mel_loss=21.3, generator_kl_loss=1.883, generator_dur_loss=1.503, generator_adv_loss=2.453, generator_feat_match_loss=3.676, over 53.00 samples.], tot_loss[discriminator_loss=2.475, discriminator_real_loss=1.218, discriminator_fake_loss=1.257, generator_loss=30.47, generator_mel_loss=21.23, generator_kl_loss=1.911, generator_dur_loss=1.508, generator_adv_loss=2.274, generator_feat_match_loss=3.55, over 492.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:33:38,453 INFO [train.py:845] (2/4) Start epoch 237
2024-02-22 23:35:24,793 INFO [train.py:471] (2/4) Epoch 237, batch 18, global_batch_idx: 8750, batch size: 56, loss[discriminator_loss=2.746, discriminator_real_loss=1.241, discriminator_fake_loss=1.505, generator_loss=29.47, generator_mel_loss=21.24, generator_kl_loss=1.838, generator_dur_loss=1.513, generator_adv_loss=1.959, generator_feat_match_loss=2.918, over 56.00 samples.], tot_loss[discriminator_loss=2.619, discriminator_real_loss=1.306, discriminator_fake_loss=1.313, generator_loss=30.04, generator_mel_loss=21.52, generator_kl_loss=1.911, generator_dur_loss=1.52, generator_adv_loss=2.065, generator_feat_match_loss=3.026, over 1279.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:37:06,669 INFO [train.py:845] (2/4) Start epoch 238
2024-02-22 23:40:05,544 INFO [train.py:471] (2/4) Epoch 238, batch 31, global_batch_idx: 8800, batch size: 52, loss[discriminator_loss=2.611, discriminator_real_loss=1.398, discriminator_fake_loss=1.213, generator_loss=30.12, generator_mel_loss=21.47, generator_kl_loss=1.91, generator_dur_loss=1.51, generator_adv_loss=2.158, generator_feat_match_loss=3.072, over 52.00 samples.], tot_loss[discriminator_loss=2.515, discriminator_real_loss=1.289, discriminator_fake_loss=1.226, generator_loss=30.73, generator_mel_loss=21.26, generator_kl_loss=1.913, generator_dur_loss=1.517, generator_adv_loss=2.362, generator_feat_match_loss=3.675, over 2272.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 32.0
2024-02-22 23:40:05,545 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 23:40:14,244 INFO [train.py:534] (2/4) Epoch 238, validation: discriminator_loss=2.484, discriminator_real_loss=1.216, discriminator_fake_loss=1.268, generator_loss=31.88, generator_mel_loss=22.77, generator_kl_loss=1.983, generator_dur_loss=1.509, generator_adv_loss=2.21, generator_feat_match_loss=3.41, over 100.00 samples.
2024-02-22 23:40:14,245 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-22 23:40:46,811 INFO [train.py:845] (2/4) Start epoch 239
2024-02-22 23:44:17,072 INFO [train.py:845] (2/4) Start epoch 240
2024-02-22 23:45:12,686 INFO [train.py:471] (2/4) Epoch 240, batch 7, global_batch_idx: 8850, batch size: 52, loss[discriminator_loss=2.602, discriminator_real_loss=1.306, discriminator_fake_loss=1.296, generator_loss=29.76, generator_mel_loss=20.96, generator_kl_loss=1.891, generator_dur_loss=1.531, generator_adv_loss=2.197, generator_feat_match_loss=3.178, over 52.00 samples.], tot_loss[discriminator_loss=2.683, discriminator_real_loss=1.334, discriminator_fake_loss=1.349, generator_loss=29.59, generator_mel_loss=21.16, generator_kl_loss=1.911, generator_dur_loss=1.509, generator_adv_loss=2.077, generator_feat_match_loss=2.933, over 655.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:47:45,234 INFO [train.py:845] (2/4) Start epoch 241
2024-02-22 23:49:51,099 INFO [train.py:471] (2/4) Epoch 241, batch 20, global_batch_idx: 8900, batch size: 95, loss[discriminator_loss=3.896, discriminator_real_loss=1.951, discriminator_fake_loss=1.945, generator_loss=31.14, generator_mel_loss=22.03, generator_kl_loss=1.953, generator_dur_loss=1.51, generator_adv_loss=2.441, generator_feat_match_loss=3.205, over 95.00 samples.], tot_loss[discriminator_loss=2.56, discriminator_real_loss=1.28, discriminator_fake_loss=1.28, generator_loss=31.17, generator_mel_loss=21.29, generator_kl_loss=1.924, generator_dur_loss=1.507, generator_adv_loss=2.47, generator_feat_match_loss=3.977, over 1718.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:51:10,271 INFO [train.py:845] (2/4) Start epoch 242
2024-02-22 23:54:22,197 INFO [train.py:471] (2/4) Epoch 242, batch 33, global_batch_idx: 8950, batch size: 49, loss[discriminator_loss=3.123, discriminator_real_loss=1.424, discriminator_fake_loss=1.699, generator_loss=28.62, generator_mel_loss=21.23, generator_kl_loss=1.886, generator_dur_loss=1.521, generator_adv_loss=1.732, generator_feat_match_loss=2.252, over 49.00 samples.], tot_loss[discriminator_loss=2.581, discriminator_real_loss=1.303, discriminator_fake_loss=1.278, generator_loss=30.13, generator_mel_loss=21, generator_kl_loss=1.899, generator_dur_loss=1.516, generator_adv_loss=2.284, generator_feat_match_loss=3.428, over 2343.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:54:37,412 INFO [train.py:845] (2/4) Start epoch 243
2024-02-22 23:58:04,645 INFO [train.py:845] (2/4) Start epoch 244
2024-02-22 23:59:10,100 INFO [train.py:471] (2/4) Epoch 244, batch 9, global_batch_idx: 9000, batch size: 50, loss[discriminator_loss=2.281, discriminator_real_loss=1.052, discriminator_fake_loss=1.23, generator_loss=30.66, generator_mel_loss=20.64, generator_kl_loss=1.771, generator_dur_loss=1.501, generator_adv_loss=2.602, generator_feat_match_loss=4.148, over 50.00 samples.], tot_loss[discriminator_loss=2.34, discriminator_real_loss=1.17, discriminator_fake_loss=1.17, generator_loss=30.92, generator_mel_loss=20.97, generator_kl_loss=1.891, generator_dur_loss=1.52, generator_adv_loss=2.408, generator_feat_match_loss=4.131, over 622.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-22 23:59:10,102 INFO [train.py:525] (2/4) Computing validation loss
2024-02-22 23:59:18,556 INFO [train.py:534] (2/4) Epoch 244, validation: discriminator_loss=2.271, discriminator_real_loss=1.164, discriminator_fake_loss=1.107, generator_loss=32.59, generator_mel_loss=22.33, generator_kl_loss=2.112, generator_dur_loss=1.507, generator_adv_loss=2.449, generator_feat_match_loss=4.194, over 100.00 samples.
2024-02-22 23:59:18,557 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 00:01:37,723 INFO [train.py:845] (2/4) Start epoch 245
2024-02-23 00:03:52,420 INFO [train.py:471] (2/4) Epoch 245, batch 22, global_batch_idx: 9050, batch size: 56, loss[discriminator_loss=2.383, discriminator_real_loss=1.242, discriminator_fake_loss=1.14, generator_loss=30.97, generator_mel_loss=20.82, generator_kl_loss=1.925, generator_dur_loss=1.504, generator_adv_loss=2.418, generator_feat_match_loss=4.305, over 56.00 samples.], tot_loss[discriminator_loss=2.506, discriminator_real_loss=1.271, discriminator_fake_loss=1.235, generator_loss=30.52, generator_mel_loss=20.97, generator_kl_loss=1.906, generator_dur_loss=1.513, generator_adv_loss=2.385, generator_feat_match_loss=3.741, over 1770.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 00:05:09,549 INFO [train.py:845] (2/4) Start epoch 246
2024-02-23 00:08:33,878 INFO [train.py:471] (2/4) Epoch 246, batch 35, global_batch_idx: 9100, batch size: 155, loss[discriminator_loss=2.602, discriminator_real_loss=1.201, discriminator_fake_loss=1.4, generator_loss=29.99, generator_mel_loss=21, generator_kl_loss=2.065, generator_dur_loss=1.475, generator_adv_loss=2.098, generator_feat_match_loss=3.352, over 155.00 samples.], tot_loss[discriminator_loss=2.493, discriminator_real_loss=1.256, discriminator_fake_loss=1.236, generator_loss=30.45, generator_mel_loss=20.92, generator_kl_loss=1.931, generator_dur_loss=1.517, generator_adv_loss=2.37, generator_feat_match_loss=3.715, over 2391.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 00:08:42,285 INFO [train.py:845] (2/4) Start epoch 247
2024-02-23 00:12:09,211 INFO [train.py:845] (2/4) Start epoch 248
2024-02-23 00:13:26,750 INFO [train.py:471] (2/4) Epoch 248, batch 11, global_batch_idx: 9150, batch size: 53, loss[discriminator_loss=2.539, discriminator_real_loss=1.354, discriminator_fake_loss=1.186, generator_loss=30.5, generator_mel_loss=21.55, generator_kl_loss=1.908, generator_dur_loss=1.524, generator_adv_loss=2.107, generator_feat_match_loss=3.406, over 53.00 samples.], tot_loss[discriminator_loss=2.52, discriminator_real_loss=1.279, discriminator_fake_loss=1.241, generator_loss=30.1, generator_mel_loss=20.96, generator_kl_loss=1.966, generator_dur_loss=1.513, generator_adv_loss=2.269, generator_feat_match_loss=3.393, over 821.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 00:15:42,662 INFO [train.py:845] (2/4) Start epoch 249
2024-02-23 00:18:04,606 INFO [train.py:471] (2/4) Epoch 249, batch 24, global_batch_idx: 9200, batch size: 76, loss[discriminator_loss=2.383, discriminator_real_loss=1.08, discriminator_fake_loss=1.303, generator_loss=31.14, generator_mel_loss=20.93, generator_kl_loss=1.885, generator_dur_loss=1.483, generator_adv_loss=2.551, generator_feat_match_loss=4.297, over 76.00 samples.], tot_loss[discriminator_loss=2.394, discriminator_real_loss=1.205, discriminator_fake_loss=1.189, generator_loss=30.85, generator_mel_loss=20.92, generator_kl_loss=1.914, generator_dur_loss=1.517, generator_adv_loss=2.424, generator_feat_match_loss=4.076, over 1595.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 32.0
2024-02-23 00:18:04,607 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 00:18:12,558 INFO [train.py:534] (2/4) Epoch 249, validation: discriminator_loss=2.333, discriminator_real_loss=1.069, discriminator_fake_loss=1.264, generator_loss=32.12, generator_mel_loss=21.78, generator_kl_loss=2.062, generator_dur_loss=1.513, generator_adv_loss=2.397, generator_feat_match_loss=4.361, over 100.00 samples.
2024-02-23 00:18:12,560 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 00:19:14,152 INFO [train.py:845] (2/4) Start epoch 250
2024-02-23 00:22:39,200 INFO [train.py:845] (2/4) Start epoch 251
2024-02-23 00:22:52,280 INFO [train.py:471] (2/4) Epoch 251, batch 0, global_batch_idx: 9250, batch size: 49, loss[discriminator_loss=2.371, discriminator_real_loss=1.347, discriminator_fake_loss=1.023, generator_loss=30.4, generator_mel_loss=20.64, generator_kl_loss=1.8, generator_dur_loss=1.51, generator_adv_loss=2.32, generator_feat_match_loss=4.129, over 49.00 samples.], tot_loss[discriminator_loss=2.371, discriminator_real_loss=1.347, discriminator_fake_loss=1.023, generator_loss=30.4, generator_mel_loss=20.64, generator_kl_loss=1.8, generator_dur_loss=1.51, generator_adv_loss=2.32, generator_feat_match_loss=4.129, over 49.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 00:26:07,663 INFO [train.py:845] (2/4) Start epoch 252
2024-02-23 00:27:28,759 INFO [train.py:471] (2/4) Epoch 252, batch 13, global_batch_idx: 9300, batch size: 102, loss[discriminator_loss=2.309, discriminator_real_loss=1.143, discriminator_fake_loss=1.167, generator_loss=30.62, generator_mel_loss=20.62, generator_kl_loss=1.971, generator_dur_loss=1.519, generator_adv_loss=2.438, generator_feat_match_loss=4.07, over 102.00 samples.], tot_loss[discriminator_loss=2.394, discriminator_real_loss=1.192, discriminator_fake_loss=1.202, generator_loss=30.57, generator_mel_loss=20.88, generator_kl_loss=1.949, generator_dur_loss=1.507, generator_adv_loss=2.37, generator_feat_match_loss=3.87, over 949.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 8.0
2024-02-23 00:29:35,988 INFO [train.py:845] (2/4) Start epoch 253
2024-02-23 00:32:22,505 INFO [train.py:471] (2/4) Epoch 253, batch 26, global_batch_idx: 9350, batch size: 55, loss[discriminator_loss=2.555, discriminator_real_loss=1.279, discriminator_fake_loss=1.275, generator_loss=29.51, generator_mel_loss=20.73, generator_kl_loss=1.828, generator_dur_loss=1.536, generator_adv_loss=2.268, generator_feat_match_loss=3.15, over 55.00 samples.], tot_loss[discriminator_loss=2.573, discriminator_real_loss=1.308, discriminator_fake_loss=1.265, generator_loss=30.59, generator_mel_loss=20.95, generator_kl_loss=1.907, generator_dur_loss=1.509, generator_adv_loss=2.391, generator_feat_match_loss=3.832, over 2117.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 8.0
2024-02-23 00:33:08,674 INFO [train.py:845] (2/4) Start epoch 254
2024-02-23 00:36:37,225 INFO [train.py:845] (2/4) Start epoch 255
2024-02-23 00:37:03,662 INFO [train.py:471] (2/4) Epoch 255, batch 2, global_batch_idx: 9400, batch size: 55, loss[discriminator_loss=2.445, discriminator_real_loss=1.186, discriminator_fake_loss=1.259, generator_loss=30.28, generator_mel_loss=20.86, generator_kl_loss=1.948, generator_dur_loss=1.556, generator_adv_loss=2.301, generator_feat_match_loss=3.613, over 55.00 samples.], tot_loss[discriminator_loss=2.416, discriminator_real_loss=1.171, discriminator_fake_loss=1.244, generator_loss=30.1, generator_mel_loss=20.71, generator_kl_loss=1.906, generator_dur_loss=1.517, generator_adv_loss=2.278, generator_feat_match_loss=3.696, over 219.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 8.0
2024-02-23 00:37:03,662 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 00:37:13,412 INFO [train.py:534] (2/4) Epoch 255, validation: discriminator_loss=2.391, discriminator_real_loss=1.144, discriminator_fake_loss=1.247, generator_loss=31.07, generator_mel_loss=21.7, generator_kl_loss=1.965, generator_dur_loss=1.499, generator_adv_loss=2.208, generator_feat_match_loss=3.697, over 100.00 samples.
2024-02-23 00:37:13,413 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 00:40:19,123 INFO [train.py:845] (2/4) Start epoch 256
2024-02-23 00:41:46,706 INFO [train.py:471] (2/4) Epoch 256, batch 15, global_batch_idx: 9450, batch size: 69, loss[discriminator_loss=2.432, discriminator_real_loss=1.188, discriminator_fake_loss=1.243, generator_loss=30.17, generator_mel_loss=20.63, generator_kl_loss=1.928, generator_dur_loss=1.5, generator_adv_loss=2.344, generator_feat_match_loss=3.768, over 69.00 samples.], tot_loss[discriminator_loss=2.452, discriminator_real_loss=1.246, discriminator_fake_loss=1.206, generator_loss=30.23, generator_mel_loss=20.67, generator_kl_loss=1.913, generator_dur_loss=1.504, generator_adv_loss=2.299, generator_feat_match_loss=3.841, over 1122.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 8.0
2024-02-23 00:43:48,334 INFO [train.py:845] (2/4) Start epoch 257
2024-02-23 00:46:39,076 INFO [train.py:471] (2/4) Epoch 257, batch 28, global_batch_idx: 9500, batch size: 110, loss[discriminator_loss=2.518, discriminator_real_loss=1.237, discriminator_fake_loss=1.28, generator_loss=30.43, generator_mel_loss=20.95, generator_kl_loss=1.972, generator_dur_loss=1.523, generator_adv_loss=2.344, generator_feat_match_loss=3.641, over 110.00 samples.], tot_loss[discriminator_loss=2.433, discriminator_real_loss=1.223, discriminator_fake_loss=1.21, generator_loss=30.73, generator_mel_loss=20.78, generator_kl_loss=1.897, generator_dur_loss=1.509, generator_adv_loss=2.429, generator_feat_match_loss=4.107, over 2317.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 8.0
2024-02-23 00:47:18,600 INFO [train.py:845] (2/4) Start epoch 258
2024-02-23 00:50:45,121 INFO [train.py:845] (2/4) Start epoch 259
2024-02-23 00:51:21,411 INFO [train.py:471] (2/4) Epoch 259, batch 4, global_batch_idx: 9550, batch size: 76, loss[discriminator_loss=2.285, discriminator_real_loss=1.098, discriminator_fake_loss=1.188, generator_loss=30.45, generator_mel_loss=20.32, generator_kl_loss=1.891, generator_dur_loss=1.519, generator_adv_loss=2.533, generator_feat_match_loss=4.18, over 76.00 samples.], tot_loss[discriminator_loss=2.299, discriminator_real_loss=1.153, discriminator_fake_loss=1.146, generator_loss=31.07, generator_mel_loss=20.73, generator_kl_loss=1.942, generator_dur_loss=1.514, generator_adv_loss=2.474, generator_feat_match_loss=4.418, over 403.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 8.0
2024-02-23 00:54:12,840 INFO [train.py:845] (2/4) Start epoch 260
2024-02-23 00:55:55,738 INFO [train.py:471] (2/4) Epoch 260, batch 17, global_batch_idx: 9600, batch size: 81, loss[discriminator_loss=2.164, discriminator_real_loss=1.063, discriminator_fake_loss=1.1, generator_loss=33.36, generator_mel_loss=21.58, generator_kl_loss=2.099, generator_dur_loss=1.521, generator_adv_loss=2.842, generator_feat_match_loss=5.324, over 81.00 samples.], tot_loss[discriminator_loss=2.405, discriminator_real_loss=1.227, discriminator_fake_loss=1.177, generator_loss=31.25, generator_mel_loss=20.91, generator_kl_loss=1.922, generator_dur_loss=1.514, generator_adv_loss=2.523, generator_feat_match_loss=4.386, over 1253.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 00:55:55,740 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 00:56:04,512 INFO [train.py:534] (2/4) Epoch 260, validation: discriminator_loss=2.182, discriminator_real_loss=1.128, discriminator_fake_loss=1.054, generator_loss=32.83, generator_mel_loss=21.43, generator_kl_loss=1.963, generator_dur_loss=1.514, generator_adv_loss=2.71, generator_feat_match_loss=5.211, over 100.00 samples.
2024-02-23 00:56:04,513 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 00:57:49,233 INFO [train.py:845] (2/4) Start epoch 261
2024-02-23 01:00:44,502 INFO [train.py:471] (2/4) Epoch 261, batch 30, global_batch_idx: 9650, batch size: 54, loss[discriminator_loss=2.426, discriminator_real_loss=1.262, discriminator_fake_loss=1.163, generator_loss=30.09, generator_mel_loss=20.52, generator_kl_loss=1.931, generator_dur_loss=1.552, generator_adv_loss=2.35, generator_feat_match_loss=3.74, over 54.00 samples.], tot_loss[discriminator_loss=2.503, discriminator_real_loss=1.304, discriminator_fake_loss=1.199, generator_loss=30.44, generator_mel_loss=20.64, generator_kl_loss=1.919, generator_dur_loss=1.512, generator_adv_loss=2.421, generator_feat_match_loss=3.951, over 2328.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 01:01:16,106 INFO [train.py:845] (2/4) Start epoch 262
2024-02-23 01:04:38,952 INFO [train.py:845] (2/4) Start epoch 263
2024-02-23 01:05:23,211 INFO [train.py:471] (2/4) Epoch 263, batch 6, global_batch_idx: 9700, batch size: 58, loss[discriminator_loss=2.537, discriminator_real_loss=1.359, discriminator_fake_loss=1.178, generator_loss=30.26, generator_mel_loss=20.32, generator_kl_loss=1.868, generator_dur_loss=1.525, generator_adv_loss=2.479, generator_feat_match_loss=4.07, over 58.00 samples.], tot_loss[discriminator_loss=2.774, discriminator_real_loss=1.545, discriminator_fake_loss=1.229, generator_loss=30.16, generator_mel_loss=20.65, generator_kl_loss=1.905, generator_dur_loss=1.518, generator_adv_loss=2.468, generator_feat_match_loss=3.622, over 459.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 01:08:08,682 INFO [train.py:845] (2/4) Start epoch 264
2024-02-23 01:09:56,049 INFO [train.py:471] (2/4) Epoch 264, batch 19, global_batch_idx: 9750, batch size: 59, loss[discriminator_loss=2.482, discriminator_real_loss=1.233, discriminator_fake_loss=1.249, generator_loss=30.16, generator_mel_loss=20.08, generator_kl_loss=1.993, generator_dur_loss=1.535, generator_adv_loss=2.387, generator_feat_match_loss=4.164, over 59.00 samples.], tot_loss[discriminator_loss=2.425, discriminator_real_loss=1.245, discriminator_fake_loss=1.18, generator_loss=30.67, generator_mel_loss=20.6, generator_kl_loss=1.913, generator_dur_loss=1.519, generator_adv_loss=2.468, generator_feat_match_loss=4.166, over 1332.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 01:11:24,764 INFO [train.py:845] (2/4) Start epoch 265
2024-02-23 01:14:36,560 INFO [train.py:471] (2/4) Epoch 265, batch 32, global_batch_idx: 9800, batch size: 51, loss[discriminator_loss=2.496, discriminator_real_loss=1.197, discriminator_fake_loss=1.299, generator_loss=29.7, generator_mel_loss=20.41, generator_kl_loss=1.934, generator_dur_loss=1.531, generator_adv_loss=2.352, generator_feat_match_loss=3.467, over 51.00 samples.], tot_loss[discriminator_loss=2.466, discriminator_real_loss=1.251, discriminator_fake_loss=1.215, generator_loss=30.5, generator_mel_loss=20.71, generator_kl_loss=1.915, generator_dur_loss=1.507, generator_adv_loss=2.415, generator_feat_match_loss=3.956, over 2524.00 samples.], cur_lr_g: 1.94e-04, cur_lr_d: 1.94e-04, grad_scale: 16.0
2024-02-23 01:14:36,562 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 01:14:45,482 INFO [train.py:534] (2/4) Epoch 265, validation: discriminator_loss=2.553, discriminator_real_loss=1.249, discriminator_fake_loss=1.304, generator_loss=30.94, generator_mel_loss=21.28, generator_kl_loss=2.113, generator_dur_loss=1.509, generator_adv_loss=2.288, generator_feat_match_loss=3.747, over 100.00 samples.
2024-02-23 01:14:45,483 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 01:15:02,880 INFO [train.py:845] (2/4) Start epoch 266
2024-02-23 01:18:32,053 INFO [train.py:845] (2/4) Start epoch 267
2024-02-23 01:19:32,452 INFO [train.py:471] (2/4) Epoch 267, batch 8, global_batch_idx: 9850, batch size: 58, loss[discriminator_loss=2.402, discriminator_real_loss=1.173, discriminator_fake_loss=1.229, generator_loss=30.16, generator_mel_loss=20.22, generator_kl_loss=1.923, generator_dur_loss=1.505, generator_adv_loss=2.432, generator_feat_match_loss=4.078, over 58.00 samples.], tot_loss[discriminator_loss=2.428, discriminator_real_loss=1.234, discriminator_fake_loss=1.195, generator_loss=30.36, generator_mel_loss=20.51, generator_kl_loss=1.887, generator_dur_loss=1.507, generator_adv_loss=2.389, generator_feat_match_loss=4.068, over 648.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 01:22:03,308 INFO [train.py:845] (2/4) Start epoch 268
2024-02-23 01:24:16,407 INFO [train.py:471] (2/4) Epoch 268, batch 21, global_batch_idx: 9900, batch size: 95, loss[discriminator_loss=2.395, discriminator_real_loss=1.185, discriminator_fake_loss=1.209, generator_loss=30.26, generator_mel_loss=20.53, generator_kl_loss=1.862, generator_dur_loss=1.526, generator_adv_loss=2.434, generator_feat_match_loss=3.904, over 95.00 samples.], tot_loss[discriminator_loss=2.43, discriminator_real_loss=1.206, discriminator_fake_loss=1.223, generator_loss=29.88, generator_mel_loss=20.44, generator_kl_loss=1.935, generator_dur_loss=1.515, generator_adv_loss=2.33, generator_feat_match_loss=3.656, over 1505.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 01:25:35,856 INFO [train.py:845] (2/4) Start epoch 269
2024-02-23 01:28:54,498 INFO [train.py:471] (2/4) Epoch 269, batch 34, global_batch_idx: 9950, batch size: 51, loss[discriminator_loss=2.328, discriminator_real_loss=1.27, discriminator_fake_loss=1.058, generator_loss=31.43, generator_mel_loss=21.2, generator_kl_loss=1.948, generator_dur_loss=1.519, generator_adv_loss=2.385, generator_feat_match_loss=4.379, over 51.00 samples.], tot_loss[discriminator_loss=2.472, discriminator_real_loss=1.268, discriminator_fake_loss=1.204, generator_loss=30.28, generator_mel_loss=20.61, generator_kl_loss=1.94, generator_dur_loss=1.506, generator_adv_loss=2.368, generator_feat_match_loss=3.86, over 2381.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 01:29:05,106 INFO [train.py:845] (2/4) Start epoch 270
2024-02-23 01:32:28,113 INFO [train.py:845] (2/4) Start epoch 271
2024-02-23 01:33:34,317 INFO [train.py:471] (2/4) Epoch 271, batch 10, global_batch_idx: 10000, batch size: 153, loss[discriminator_loss=2.463, discriminator_real_loss=1.396, discriminator_fake_loss=1.067, generator_loss=31.02, generator_mel_loss=20.6, generator_kl_loss=2.064, generator_dur_loss=1.501, generator_adv_loss=2.393, generator_feat_match_loss=4.465, over 153.00 samples.], tot_loss[discriminator_loss=2.397, discriminator_real_loss=1.221, discriminator_fake_loss=1.176, generator_loss=30.53, generator_mel_loss=20.59, generator_kl_loss=1.952, generator_dur_loss=1.509, generator_adv_loss=2.396, generator_feat_match_loss=4.088, over 796.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 32.0
2024-02-23 01:33:34,318 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 01:33:42,146 INFO [train.py:534] (2/4) Epoch 271, validation: discriminator_loss=2.414, discriminator_real_loss=1.118, discriminator_fake_loss=1.296, generator_loss=31.98, generator_mel_loss=21.33, generator_kl_loss=2.093, generator_dur_loss=1.499, generator_adv_loss=2.373, generator_feat_match_loss=4.684, over 100.00 samples.
2024-02-23 01:33:42,147 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 01:36:03,497 INFO [train.py:845] (2/4) Start epoch 272
2024-02-23 01:38:17,556 INFO [train.py:471] (2/4) Epoch 272, batch 23, global_batch_idx: 10050, batch size: 64, loss[discriminator_loss=2.367, discriminator_real_loss=1.127, discriminator_fake_loss=1.239, generator_loss=30.08, generator_mel_loss=20.12, generator_kl_loss=1.882, generator_dur_loss=1.516, generator_adv_loss=2.381, generator_feat_match_loss=4.18, over 64.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.223, discriminator_fake_loss=1.177, generator_loss=30.51, generator_mel_loss=20.45, generator_kl_loss=1.941, generator_dur_loss=1.507, generator_adv_loss=2.459, generator_feat_match_loss=4.155, over 1749.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 01:39:30,949 INFO [train.py:845] (2/4) Start epoch 273
2024-02-23 01:43:04,226 INFO [train.py:471] (2/4) Epoch 273, batch 36, global_batch_idx: 10100, batch size: 56, loss[discriminator_loss=2.547, discriminator_real_loss=1.239, discriminator_fake_loss=1.309, generator_loss=29.47, generator_mel_loss=20.47, generator_kl_loss=1.866, generator_dur_loss=1.524, generator_adv_loss=2.205, generator_feat_match_loss=3.402, over 56.00 samples.], tot_loss[discriminator_loss=2.572, discriminator_real_loss=1.352, discriminator_fake_loss=1.22, generator_loss=30.42, generator_mel_loss=20.63, generator_kl_loss=1.923, generator_dur_loss=1.508, generator_adv_loss=2.473, generator_feat_match_loss=3.877, over 2681.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 01:43:04,670 INFO [train.py:845] (2/4) Start epoch 274
2024-02-23 01:46:33,213 INFO [train.py:845] (2/4) Start epoch 275
2024-02-23 01:47:52,093 INFO [train.py:471] (2/4) Epoch 275, batch 12, global_batch_idx: 10150, batch size: 63, loss[discriminator_loss=2.293, discriminator_real_loss=1.07, discriminator_fake_loss=1.223, generator_loss=30.96, generator_mel_loss=20.59, generator_kl_loss=1.936, generator_dur_loss=1.541, generator_adv_loss=2.43, generator_feat_match_loss=4.465, over 63.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.195, discriminator_fake_loss=1.206, generator_loss=30.41, generator_mel_loss=20.62, generator_kl_loss=1.924, generator_dur_loss=1.511, generator_adv_loss=2.392, generator_feat_match_loss=3.959, over 949.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 01:50:03,735 INFO [train.py:845] (2/4) Start epoch 276
2024-02-23 01:52:23,960 INFO [train.py:471] (2/4) Epoch 276, batch 25, global_batch_idx: 10200, batch size: 59, loss[discriminator_loss=2.506, discriminator_real_loss=1.324, discriminator_fake_loss=1.182, generator_loss=30.92, generator_mel_loss=20.77, generator_kl_loss=1.89, generator_dur_loss=1.507, generator_adv_loss=2.604, generator_feat_match_loss=4.148, over 59.00 samples.], tot_loss[discriminator_loss=2.376, discriminator_real_loss=1.193, discriminator_fake_loss=1.183, generator_loss=30.44, generator_mel_loss=20.47, generator_kl_loss=1.941, generator_dur_loss=1.508, generator_adv_loss=2.424, generator_feat_match_loss=4.1, over 1860.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 01:52:23,961 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 01:52:32,155 INFO [train.py:534] (2/4) Epoch 276, validation: discriminator_loss=2.562, discriminator_real_loss=1.431, discriminator_fake_loss=1.131, generator_loss=31.64, generator_mel_loss=21.27, generator_kl_loss=2.058, generator_dur_loss=1.501, generator_adv_loss=2.504, generator_feat_match_loss=4.301, over 100.00 samples.
2024-02-23 01:52:32,156 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 01:53:38,851 INFO [train.py:845] (2/4) Start epoch 277
2024-02-23 01:57:09,420 INFO [train.py:845] (2/4) Start epoch 278
2024-02-23 01:57:30,346 INFO [train.py:471] (2/4) Epoch 278, batch 1, global_batch_idx: 10250, batch size: 126, loss[discriminator_loss=2.664, discriminator_real_loss=1.363, discriminator_fake_loss=1.302, generator_loss=30.36, generator_mel_loss=21.01, generator_kl_loss=1.865, generator_dur_loss=1.483, generator_adv_loss=2.268, generator_feat_match_loss=3.736, over 126.00 samples.], tot_loss[discriminator_loss=2.7, discriminator_real_loss=1.498, discriminator_fake_loss=1.203, generator_loss=30.29, generator_mel_loss=20.92, generator_kl_loss=1.873, generator_dur_loss=1.492, generator_adv_loss=2.275, generator_feat_match_loss=3.736, over 175.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:00:36,682 INFO [train.py:845] (2/4) Start epoch 279
2024-02-23 02:02:06,812 INFO [train.py:471] (2/4) Epoch 279, batch 14, global_batch_idx: 10300, batch size: 85, loss[discriminator_loss=2.424, discriminator_real_loss=1.18, discriminator_fake_loss=1.244, generator_loss=29.57, generator_mel_loss=20.01, generator_kl_loss=1.925, generator_dur_loss=1.508, generator_adv_loss=2.291, generator_feat_match_loss=3.838, over 85.00 samples.], tot_loss[discriminator_loss=2.408, discriminator_real_loss=1.21, discriminator_fake_loss=1.198, generator_loss=30.13, generator_mel_loss=20.34, generator_kl_loss=1.941, generator_dur_loss=1.508, generator_adv_loss=2.39, generator_feat_match_loss=3.956, over 1216.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:04:08,324 INFO [train.py:845] (2/4) Start epoch 280
2024-02-23 02:06:47,190 INFO [train.py:471] (2/4) Epoch 280, batch 27, global_batch_idx: 10350, batch size: 126, loss[discriminator_loss=2.537, discriminator_real_loss=1.146, discriminator_fake_loss=1.391, generator_loss=30.31, generator_mel_loss=20.57, generator_kl_loss=1.878, generator_dur_loss=1.508, generator_adv_loss=2.227, generator_feat_match_loss=4.133, over 126.00 samples.], tot_loss[discriminator_loss=2.536, discriminator_real_loss=1.328, discriminator_fake_loss=1.208, generator_loss=30.69, generator_mel_loss=20.46, generator_kl_loss=1.949, generator_dur_loss=1.507, generator_adv_loss=2.548, generator_feat_match_loss=4.232, over 2069.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:07:34,465 INFO [train.py:845] (2/4) Start epoch 281
2024-02-23 02:11:05,398 INFO [train.py:845] (2/4) Start epoch 282
2024-02-23 02:11:36,271 INFO [train.py:471] (2/4) Epoch 282, batch 3, global_batch_idx: 10400, batch size: 79, loss[discriminator_loss=2.455, discriminator_real_loss=1.258, discriminator_fake_loss=1.197, generator_loss=29.44, generator_mel_loss=20.5, generator_kl_loss=1.828, generator_dur_loss=1.502, generator_adv_loss=2.314, generator_feat_match_loss=3.297, over 79.00 samples.], tot_loss[discriminator_loss=2.458, discriminator_real_loss=1.208, discriminator_fake_loss=1.25, generator_loss=29.77, generator_mel_loss=20.47, generator_kl_loss=1.882, generator_dur_loss=1.516, generator_adv_loss=2.309, generator_feat_match_loss=3.592, over 347.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 32.0
2024-02-23 02:11:36,272 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 02:11:44,146 INFO [train.py:534] (2/4) Epoch 282, validation: discriminator_loss=2.473, discriminator_real_loss=1.178, discriminator_fake_loss=1.295, generator_loss=30.98, generator_mel_loss=21.46, generator_kl_loss=1.989, generator_dur_loss=1.509, generator_adv_loss=2.29, generator_feat_match_loss=3.737, over 100.00 samples.
2024-02-23 02:11:44,147 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 02:14:39,423 INFO [train.py:845] (2/4) Start epoch 283
2024-02-23 02:16:09,272 INFO [train.py:471] (2/4) Epoch 283, batch 16, global_batch_idx: 10450, batch size: 85, loss[discriminator_loss=2.295, discriminator_real_loss=1.229, discriminator_fake_loss=1.065, generator_loss=30.35, generator_mel_loss=20.01, generator_kl_loss=1.913, generator_dur_loss=1.488, generator_adv_loss=2.453, generator_feat_match_loss=4.488, over 85.00 samples.], tot_loss[discriminator_loss=2.395, discriminator_real_loss=1.217, discriminator_fake_loss=1.178, generator_loss=30.23, generator_mel_loss=20.25, generator_kl_loss=1.919, generator_dur_loss=1.508, generator_adv_loss=2.437, generator_feat_match_loss=4.111, over 1094.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:18:02,808 INFO [train.py:845] (2/4) Start epoch 284
2024-02-23 02:20:55,886 INFO [train.py:471] (2/4) Epoch 284, batch 29, global_batch_idx: 10500, batch size: 58, loss[discriminator_loss=2.305, discriminator_real_loss=1.192, discriminator_fake_loss=1.113, generator_loss=30.66, generator_mel_loss=20.14, generator_kl_loss=1.944, generator_dur_loss=1.516, generator_adv_loss=2.498, generator_feat_match_loss=4.562, over 58.00 samples.], tot_loss[discriminator_loss=2.38, discriminator_real_loss=1.208, discriminator_fake_loss=1.172, generator_loss=30.43, generator_mel_loss=20.31, generator_kl_loss=1.954, generator_dur_loss=1.508, generator_adv_loss=2.444, generator_feat_match_loss=4.206, over 2032.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:21:38,924 INFO [train.py:845] (2/4) Start epoch 285
2024-02-23 02:25:12,295 INFO [train.py:845] (2/4) Start epoch 286
2024-02-23 02:25:51,293 INFO [train.py:471] (2/4) Epoch 286, batch 5, global_batch_idx: 10550, batch size: 79, loss[discriminator_loss=2.312, discriminator_real_loss=1.123, discriminator_fake_loss=1.188, generator_loss=30.09, generator_mel_loss=19.86, generator_kl_loss=1.928, generator_dur_loss=1.51, generator_adv_loss=2.436, generator_feat_match_loss=4.359, over 79.00 samples.], tot_loss[discriminator_loss=2.369, discriminator_real_loss=1.184, discriminator_fake_loss=1.185, generator_loss=30.23, generator_mel_loss=20.12, generator_kl_loss=1.911, generator_dur_loss=1.508, generator_adv_loss=2.423, generator_feat_match_loss=4.268, over 431.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:28:44,334 INFO [train.py:845] (2/4) Start epoch 287
2024-02-23 02:30:36,751 INFO [train.py:471] (2/4) Epoch 287, batch 18, global_batch_idx: 10600, batch size: 53, loss[discriminator_loss=2.271, discriminator_real_loss=1.333, discriminator_fake_loss=0.939, generator_loss=32.67, generator_mel_loss=20.97, generator_kl_loss=1.896, generator_dur_loss=1.538, generator_adv_loss=2.68, generator_feat_match_loss=5.582, over 53.00 samples.], tot_loss[discriminator_loss=2.407, discriminator_real_loss=1.231, discriminator_fake_loss=1.176, generator_loss=30.73, generator_mel_loss=20.59, generator_kl_loss=1.932, generator_dur_loss=1.507, generator_adv_loss=2.459, generator_feat_match_loss=4.242, over 1420.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:30:36,753 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 02:30:44,617 INFO [train.py:534] (2/4) Epoch 287, validation: discriminator_loss=2.075, discriminator_real_loss=0.9959, discriminator_fake_loss=1.079, generator_loss=33.42, generator_mel_loss=21.64, generator_kl_loss=2.035, generator_dur_loss=1.502, generator_adv_loss=2.61, generator_feat_match_loss=5.626, over 100.00 samples.
2024-02-23 02:30:44,618 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 02:32:20,577 INFO [train.py:845] (2/4) Start epoch 288
2024-02-23 02:35:18,396 INFO [train.py:471] (2/4) Epoch 288, batch 31, global_batch_idx: 10650, batch size: 126, loss[discriminator_loss=2.336, discriminator_real_loss=1.114, discriminator_fake_loss=1.221, generator_loss=30.72, generator_mel_loss=20.65, generator_kl_loss=1.974, generator_dur_loss=1.501, generator_adv_loss=2.385, generator_feat_match_loss=4.215, over 126.00 samples.], tot_loss[discriminator_loss=2.415, discriminator_real_loss=1.201, discriminator_fake_loss=1.214, generator_loss=29.89, generator_mel_loss=20.3, generator_kl_loss=1.927, generator_dur_loss=1.511, generator_adv_loss=2.342, generator_feat_match_loss=3.813, over 2161.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:35:49,186 INFO [train.py:845] (2/4) Start epoch 289
2024-02-23 02:39:17,945 INFO [train.py:845] (2/4) Start epoch 290
2024-02-23 02:40:14,574 INFO [train.py:471] (2/4) Epoch 290, batch 7, global_batch_idx: 10700, batch size: 110, loss[discriminator_loss=2.408, discriminator_real_loss=1.173, discriminator_fake_loss=1.235, generator_loss=30.24, generator_mel_loss=20.64, generator_kl_loss=1.911, generator_dur_loss=1.496, generator_adv_loss=2.363, generator_feat_match_loss=3.824, over 110.00 samples.], tot_loss[discriminator_loss=2.462, discriminator_real_loss=1.263, discriminator_fake_loss=1.199, generator_loss=30.29, generator_mel_loss=20.53, generator_kl_loss=1.919, generator_dur_loss=1.5, generator_adv_loss=2.4, generator_feat_match_loss=3.943, over 680.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:42:46,365 INFO [train.py:845] (2/4) Start epoch 291
2024-02-23 02:44:48,833 INFO [train.py:471] (2/4) Epoch 291, batch 20, global_batch_idx: 10750, batch size: 69, loss[discriminator_loss=2.438, discriminator_real_loss=1.065, discriminator_fake_loss=1.371, generator_loss=29.79, generator_mel_loss=19.78, generator_kl_loss=1.98, generator_dur_loss=1.486, generator_adv_loss=2.338, generator_feat_match_loss=4.211, over 69.00 samples.], tot_loss[discriminator_loss=2.465, discriminator_real_loss=1.261, discriminator_fake_loss=1.204, generator_loss=30.8, generator_mel_loss=20.43, generator_kl_loss=1.925, generator_dur_loss=1.508, generator_adv_loss=2.544, generator_feat_match_loss=4.387, over 1426.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 02:46:13,008 INFO [train.py:845] (2/4) Start epoch 292
2024-02-23 02:49:23,796 INFO [train.py:471] (2/4) Epoch 292, batch 33, global_batch_idx: 10800, batch size: 76, loss[discriminator_loss=2.441, discriminator_real_loss=1.255, discriminator_fake_loss=1.187, generator_loss=29.68, generator_mel_loss=20.12, generator_kl_loss=1.944, generator_dur_loss=1.512, generator_adv_loss=2.332, generator_feat_match_loss=3.771, over 76.00 samples.], tot_loss[discriminator_loss=2.461, discriminator_real_loss=1.263, discriminator_fake_loss=1.198, generator_loss=30.41, generator_mel_loss=20.39, generator_kl_loss=1.929, generator_dur_loss=1.507, generator_adv_loss=2.46, generator_feat_match_loss=4.124, over 2566.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 32.0
2024-02-23 02:49:23,797 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 02:49:31,839 INFO [train.py:534] (2/4) Epoch 292, validation: discriminator_loss=2.433, discriminator_real_loss=1.199, discriminator_fake_loss=1.234, generator_loss=30.61, generator_mel_loss=21.11, generator_kl_loss=2.047, generator_dur_loss=1.493, generator_adv_loss=2.275, generator_feat_match_loss=3.682, over 100.00 samples.
2024-02-23 02:49:31,840 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 02:49:51,987 INFO [train.py:845] (2/4) Start epoch 293
2024-02-23 02:53:23,747 INFO [train.py:845] (2/4) Start epoch 294
2024-02-23 02:54:24,994 INFO [train.py:471] (2/4) Epoch 294, batch 9, global_batch_idx: 10850, batch size: 73, loss[discriminator_loss=2.727, discriminator_real_loss=1.45, discriminator_fake_loss=1.275, generator_loss=30.09, generator_mel_loss=20.43, generator_kl_loss=1.887, generator_dur_loss=1.493, generator_adv_loss=2.561, generator_feat_match_loss=3.719, over 73.00 samples.], tot_loss[discriminator_loss=2.347, discriminator_real_loss=1.159, discriminator_fake_loss=1.188, generator_loss=30.64, generator_mel_loss=20.25, generator_kl_loss=1.903, generator_dur_loss=1.504, generator_adv_loss=2.521, generator_feat_match_loss=4.465, over 707.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 32.0
2024-02-23 02:56:52,444 INFO [train.py:845] (2/4) Start epoch 295
2024-02-23 02:58:57,705 INFO [train.py:471] (2/4) Epoch 295, batch 22, global_batch_idx: 10900, batch size: 110, loss[discriminator_loss=2.383, discriminator_real_loss=1.228, discriminator_fake_loss=1.156, generator_loss=30.45, generator_mel_loss=20.77, generator_kl_loss=2.001, generator_dur_loss=1.5, generator_adv_loss=2.357, generator_feat_match_loss=3.816, over 110.00 samples.], tot_loss[discriminator_loss=2.504, discriminator_real_loss=1.286, discriminator_fake_loss=1.218, generator_loss=29.67, generator_mel_loss=20.23, generator_kl_loss=1.924, generator_dur_loss=1.503, generator_adv_loss=2.374, generator_feat_match_loss=3.644, over 1704.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:00:16,026 INFO [train.py:845] (2/4) Start epoch 296
2024-02-23 03:03:38,371 INFO [train.py:471] (2/4) Epoch 296, batch 35, global_batch_idx: 10950, batch size: 65, loss[discriminator_loss=2.299, discriminator_real_loss=1.16, discriminator_fake_loss=1.139, generator_loss=29.99, generator_mel_loss=19.84, generator_kl_loss=1.829, generator_dur_loss=1.48, generator_adv_loss=2.508, generator_feat_match_loss=4.336, over 65.00 samples.], tot_loss[discriminator_loss=2.474, discriminator_real_loss=1.251, discriminator_fake_loss=1.224, generator_loss=30.12, generator_mel_loss=20.22, generator_kl_loss=1.91, generator_dur_loss=1.503, generator_adv_loss=2.433, generator_feat_match_loss=4.047, over 2598.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:03:44,688 INFO [train.py:845] (2/4) Start epoch 297
2024-02-23 03:07:13,724 INFO [train.py:845] (2/4) Start epoch 298
2024-02-23 03:08:22,455 INFO [train.py:471] (2/4) Epoch 298, batch 11, global_batch_idx: 11000, batch size: 73, loss[discriminator_loss=2.354, discriminator_real_loss=1.074, discriminator_fake_loss=1.279, generator_loss=30.24, generator_mel_loss=20.02, generator_kl_loss=1.978, generator_dur_loss=1.499, generator_adv_loss=2.443, generator_feat_match_loss=4.297, over 73.00 samples.], tot_loss[discriminator_loss=2.381, discriminator_real_loss=1.18, discriminator_fake_loss=1.201, generator_loss=30.3, generator_mel_loss=20.3, generator_kl_loss=1.936, generator_dur_loss=1.504, generator_adv_loss=2.429, generator_feat_match_loss=4.13, over 893.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:08:22,456 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 03:08:30,941 INFO [train.py:534] (2/4) Epoch 298, validation: discriminator_loss=2.331, discriminator_real_loss=1.139, discriminator_fake_loss=1.193, generator_loss=32.38, generator_mel_loss=21.65, generator_kl_loss=2.029, generator_dur_loss=1.503, generator_adv_loss=2.438, generator_feat_match_loss=4.753, over 100.00 samples.
2024-02-23 03:08:30,942 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 03:10:45,287 INFO [train.py:845] (2/4) Start epoch 299
2024-02-23 03:13:11,221 INFO [train.py:471] (2/4) Epoch 299, batch 24, global_batch_idx: 11050, batch size: 53, loss[discriminator_loss=2.465, discriminator_real_loss=1.362, discriminator_fake_loss=1.102, generator_loss=30.38, generator_mel_loss=20.12, generator_kl_loss=1.929, generator_dur_loss=1.543, generator_adv_loss=2.574, generator_feat_match_loss=4.215, over 53.00 samples.], tot_loss[discriminator_loss=2.412, discriminator_real_loss=1.221, discriminator_fake_loss=1.191, generator_loss=30.26, generator_mel_loss=20.21, generator_kl_loss=1.93, generator_dur_loss=1.508, generator_adv_loss=2.444, generator_feat_match_loss=4.169, over 1817.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:14:17,632 INFO [train.py:845] (2/4) Start epoch 300
2024-02-23 03:17:46,547 INFO [train.py:845] (2/4) Start epoch 301
2024-02-23 03:18:01,297 INFO [train.py:471] (2/4) Epoch 301, batch 0, global_batch_idx: 11100, batch size: 71, loss[discriminator_loss=2.459, discriminator_real_loss=1.275, discriminator_fake_loss=1.184, generator_loss=29.56, generator_mel_loss=20.04, generator_kl_loss=1.912, generator_dur_loss=1.502, generator_adv_loss=2.363, generator_feat_match_loss=3.742, over 71.00 samples.], tot_loss[discriminator_loss=2.459, discriminator_real_loss=1.275, discriminator_fake_loss=1.184, generator_loss=29.56, generator_mel_loss=20.04, generator_kl_loss=1.912, generator_dur_loss=1.502, generator_adv_loss=2.363, generator_feat_match_loss=3.742, over 71.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:21:12,884 INFO [train.py:845] (2/4) Start epoch 302
2024-02-23 03:22:32,048 INFO [train.py:471] (2/4) Epoch 302, batch 13, global_batch_idx: 11150, batch size: 90, loss[discriminator_loss=2.273, discriminator_real_loss=1.092, discriminator_fake_loss=1.181, generator_loss=29.81, generator_mel_loss=19.97, generator_kl_loss=1.844, generator_dur_loss=1.501, generator_adv_loss=2.346, generator_feat_match_loss=4.148, over 90.00 samples.], tot_loss[discriminator_loss=2.388, discriminator_real_loss=1.168, discriminator_fake_loss=1.219, generator_loss=29.89, generator_mel_loss=20.18, generator_kl_loss=1.92, generator_dur_loss=1.504, generator_adv_loss=2.331, generator_feat_match_loss=3.954, over 1062.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:24:40,859 INFO [train.py:845] (2/4) Start epoch 303
2024-02-23 03:27:15,412 INFO [train.py:471] (2/4) Epoch 303, batch 26, global_batch_idx: 11200, batch size: 59, loss[discriminator_loss=2.633, discriminator_real_loss=1.443, discriminator_fake_loss=1.19, generator_loss=30.6, generator_mel_loss=20.39, generator_kl_loss=2.05, generator_dur_loss=1.526, generator_adv_loss=2.584, generator_feat_match_loss=4.047, over 59.00 samples.], tot_loss[discriminator_loss=2.353, discriminator_real_loss=1.185, discriminator_fake_loss=1.168, generator_loss=30.56, generator_mel_loss=20.34, generator_kl_loss=1.951, generator_dur_loss=1.503, generator_adv_loss=2.463, generator_feat_match_loss=4.3, over 2122.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 32.0
2024-02-23 03:27:15,413 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 03:27:24,543 INFO [train.py:534] (2/4) Epoch 303, validation: discriminator_loss=2.704, discriminator_real_loss=1.501, discriminator_fake_loss=1.202, generator_loss=31.55, generator_mel_loss=21.15, generator_kl_loss=2.052, generator_dur_loss=1.494, generator_adv_loss=2.451, generator_feat_match_loss=4.404, over 100.00 samples.
2024-02-23 03:27:24,544 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 03:28:16,409 INFO [train.py:845] (2/4) Start epoch 304
2024-02-23 03:31:48,739 INFO [train.py:845] (2/4) Start epoch 305
2024-02-23 03:32:13,374 INFO [train.py:471] (2/4) Epoch 305, batch 2, global_batch_idx: 11250, batch size: 54, loss[discriminator_loss=2.459, discriminator_real_loss=1.237, discriminator_fake_loss=1.222, generator_loss=29.21, generator_mel_loss=20.06, generator_kl_loss=1.857, generator_dur_loss=1.515, generator_adv_loss=2.258, generator_feat_match_loss=3.516, over 54.00 samples.], tot_loss[discriminator_loss=2.485, discriminator_real_loss=1.239, discriminator_fake_loss=1.245, generator_loss=29.63, generator_mel_loss=20.22, generator_kl_loss=1.964, generator_dur_loss=1.501, generator_adv_loss=2.287, generator_feat_match_loss=3.662, over 247.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:35:13,662 INFO [train.py:845] (2/4) Start epoch 306
2024-02-23 03:36:44,480 INFO [train.py:471] (2/4) Epoch 306, batch 15, global_batch_idx: 11300, batch size: 71, loss[discriminator_loss=2.355, discriminator_real_loss=1.158, discriminator_fake_loss=1.198, generator_loss=30.04, generator_mel_loss=20.07, generator_kl_loss=1.934, generator_dur_loss=1.509, generator_adv_loss=2.426, generator_feat_match_loss=4.102, over 71.00 samples.], tot_loss[discriminator_loss=2.371, discriminator_real_loss=1.17, discriminator_fake_loss=1.201, generator_loss=30.39, generator_mel_loss=20.42, generator_kl_loss=1.939, generator_dur_loss=1.509, generator_adv_loss=2.386, generator_feat_match_loss=4.137, over 1110.00 samples.], cur_lr_g: 1.93e-04, cur_lr_d: 1.93e-04, grad_scale: 16.0
2024-02-23 03:38:39,713 INFO [train.py:845] (2/4) Start epoch 307
2024-02-23 03:41:21,385 INFO [train.py:471] (2/4) Epoch 307, batch 28, global_batch_idx: 11350, batch size: 101, loss[discriminator_loss=2.422, discriminator_real_loss=1.156, discriminator_fake_loss=1.265, generator_loss=30.51, generator_mel_loss=20.23, generator_kl_loss=2.026, generator_dur_loss=1.478, generator_adv_loss=2.527, generator_feat_match_loss=4.242, over 101.00 samples.], tot_loss[discriminator_loss=2.372, discriminator_real_loss=1.196, discriminator_fake_loss=1.176, generator_loss=30.39, generator_mel_loss=20.2, generator_kl_loss=1.921, generator_dur_loss=1.513, generator_adv_loss=2.463, generator_feat_match_loss=4.292, over 1974.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 03:42:07,441 INFO [train.py:845] (2/4) Start epoch 308
2024-02-23 03:45:34,203 INFO [train.py:845] (2/4) Start epoch 309
2024-02-23 03:46:06,301 INFO [train.py:471] (2/4) Epoch 309, batch 4, global_batch_idx: 11400, batch size: 73, loss[discriminator_loss=2.426, discriminator_real_loss=1.354, discriminator_fake_loss=1.072, generator_loss=30.71, generator_mel_loss=20.81, generator_kl_loss=1.893, generator_dur_loss=1.5, generator_adv_loss=2.355, generator_feat_match_loss=4.145, over 73.00 samples.], tot_loss[discriminator_loss=2.372, discriminator_real_loss=1.215, discriminator_fake_loss=1.158, generator_loss=30.49, generator_mel_loss=20.33, generator_kl_loss=1.912, generator_dur_loss=1.505, generator_adv_loss=2.467, generator_feat_match_loss=4.275, over 348.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 03:46:06,302 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 03:46:14,931 INFO [train.py:534] (2/4) Epoch 309, validation: discriminator_loss=2.501, discriminator_real_loss=1.044, discriminator_fake_loss=1.458, generator_loss=30.64, generator_mel_loss=20.79, generator_kl_loss=2.017, generator_dur_loss=1.505, generator_adv_loss=2.224, generator_feat_match_loss=4.102, over 100.00 samples.
2024-02-23 03:46:14,933 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 03:49:14,620 INFO [train.py:845] (2/4) Start epoch 310
2024-02-23 03:51:06,715 INFO [train.py:471] (2/4) Epoch 310, batch 17, global_batch_idx: 11450, batch size: 71, loss[discriminator_loss=2.863, discriminator_real_loss=1.192, discriminator_fake_loss=1.671, generator_loss=30.07, generator_mel_loss=20.57, generator_kl_loss=1.92, generator_dur_loss=1.51, generator_adv_loss=2.994, generator_feat_match_loss=3.078, over 71.00 samples.], tot_loss[discriminator_loss=2.605, discriminator_real_loss=1.325, discriminator_fake_loss=1.28, generator_loss=29.75, generator_mel_loss=20.39, generator_kl_loss=1.928, generator_dur_loss=1.502, generator_adv_loss=2.292, generator_feat_match_loss=3.636, over 1349.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 03:52:43,137 INFO [train.py:845] (2/4) Start epoch 311
2024-02-23 03:55:32,691 INFO [train.py:471] (2/4) Epoch 311, batch 30, global_batch_idx: 11500, batch size: 153, loss[discriminator_loss=2.414, discriminator_real_loss=1.183, discriminator_fake_loss=1.23, generator_loss=30.01, generator_mel_loss=20.24, generator_kl_loss=1.982, generator_dur_loss=1.501, generator_adv_loss=2.305, generator_feat_match_loss=3.984, over 153.00 samples.], tot_loss[discriminator_loss=2.46, discriminator_real_loss=1.246, discriminator_fake_loss=1.214, generator_loss=30.1, generator_mel_loss=20.27, generator_kl_loss=1.947, generator_dur_loss=1.5, generator_adv_loss=2.39, generator_feat_match_loss=3.987, over 2416.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 03:56:07,555 INFO [train.py:845] (2/4) Start epoch 312
2024-02-23 03:59:33,550 INFO [train.py:845] (2/4) Start epoch 313
2024-02-23 04:00:20,459 INFO [train.py:471] (2/4) Epoch 313, batch 6, global_batch_idx: 11550, batch size: 53, loss[discriminator_loss=2.381, discriminator_real_loss=1.19, discriminator_fake_loss=1.19, generator_loss=30.27, generator_mel_loss=20.59, generator_kl_loss=1.878, generator_dur_loss=1.494, generator_adv_loss=2.34, generator_feat_match_loss=3.969, over 53.00 samples.], tot_loss[discriminator_loss=2.436, discriminator_real_loss=1.219, discriminator_fake_loss=1.217, generator_loss=29.89, generator_mel_loss=20.38, generator_kl_loss=1.958, generator_dur_loss=1.501, generator_adv_loss=2.318, generator_feat_match_loss=3.725, over 522.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:03:02,326 INFO [train.py:845] (2/4) Start epoch 314
2024-02-23 04:05:05,256 INFO [train.py:471] (2/4) Epoch 314, batch 19, global_batch_idx: 11600, batch size: 101, loss[discriminator_loss=2.57, discriminator_real_loss=1.18, discriminator_fake_loss=1.392, generator_loss=31.1, generator_mel_loss=20.48, generator_kl_loss=2.008, generator_dur_loss=1.492, generator_adv_loss=2.727, generator_feat_match_loss=4.395, over 101.00 samples.], tot_loss[discriminator_loss=2.406, discriminator_real_loss=1.201, discriminator_fake_loss=1.205, generator_loss=30.38, generator_mel_loss=20.18, generator_kl_loss=1.95, generator_dur_loss=1.502, generator_adv_loss=2.484, generator_feat_match_loss=4.258, over 1538.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 32.0
2024-02-23 04:05:05,257 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 04:05:13,216 INFO [train.py:534] (2/4) Epoch 314, validation: discriminator_loss=2.443, discriminator_real_loss=1.45, discriminator_fake_loss=0.9932, generator_loss=31.98, generator_mel_loss=21.2, generator_kl_loss=2.121, generator_dur_loss=1.5, generator_adv_loss=2.658, generator_feat_match_loss=4.501, over 100.00 samples.
2024-02-23 04:05:13,217 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 04:06:39,613 INFO [train.py:845] (2/4) Start epoch 315
2024-02-23 04:09:54,971 INFO [train.py:471] (2/4) Epoch 315, batch 32, global_batch_idx: 11650, batch size: 76, loss[discriminator_loss=2.422, discriminator_real_loss=1.185, discriminator_fake_loss=1.238, generator_loss=29.95, generator_mel_loss=20.12, generator_kl_loss=1.955, generator_dur_loss=1.498, generator_adv_loss=2.398, generator_feat_match_loss=3.975, over 76.00 samples.], tot_loss[discriminator_loss=2.392, discriminator_real_loss=1.2, discriminator_fake_loss=1.192, generator_loss=29.8, generator_mel_loss=19.92, generator_kl_loss=1.928, generator_dur_loss=1.499, generator_adv_loss=2.4, generator_feat_match_loss=4.049, over 2271.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:10:14,203 INFO [train.py:845] (2/4) Start epoch 316
2024-02-23 04:13:46,227 INFO [train.py:845] (2/4) Start epoch 317
2024-02-23 04:14:47,717 INFO [train.py:471] (2/4) Epoch 317, batch 8, global_batch_idx: 11700, batch size: 55, loss[discriminator_loss=2.334, discriminator_real_loss=1.188, discriminator_fake_loss=1.146, generator_loss=30.34, generator_mel_loss=20.46, generator_kl_loss=1.932, generator_dur_loss=1.513, generator_adv_loss=2.4, generator_feat_match_loss=4.031, over 55.00 samples.], tot_loss[discriminator_loss=2.409, discriminator_real_loss=1.26, discriminator_fake_loss=1.149, generator_loss=30.86, generator_mel_loss=20.49, generator_kl_loss=1.951, generator_dur_loss=1.515, generator_adv_loss=2.53, generator_feat_match_loss=4.379, over 549.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:17:12,963 INFO [train.py:845] (2/4) Start epoch 318
2024-02-23 04:19:24,895 INFO [train.py:471] (2/4) Epoch 318, batch 21, global_batch_idx: 11750, batch size: 101, loss[discriminator_loss=2.625, discriminator_real_loss=1.158, discriminator_fake_loss=1.467, generator_loss=30.92, generator_mel_loss=20.99, generator_kl_loss=1.891, generator_dur_loss=1.507, generator_adv_loss=2.113, generator_feat_match_loss=4.418, over 101.00 samples.], tot_loss[discriminator_loss=2.493, discriminator_real_loss=1.277, discriminator_fake_loss=1.216, generator_loss=31.19, generator_mel_loss=20.37, generator_kl_loss=1.947, generator_dur_loss=1.496, generator_adv_loss=2.646, generator_feat_match_loss=4.724, over 1762.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:20:41,719 INFO [train.py:845] (2/4) Start epoch 319
2024-02-23 04:24:05,789 INFO [train.py:471] (2/4) Epoch 319, batch 34, global_batch_idx: 11800, batch size: 101, loss[discriminator_loss=2.342, discriminator_real_loss=1.242, discriminator_fake_loss=1.1, generator_loss=29.76, generator_mel_loss=19.75, generator_kl_loss=1.897, generator_dur_loss=1.459, generator_adv_loss=2.291, generator_feat_match_loss=4.367, over 101.00 samples.], tot_loss[discriminator_loss=2.43, discriminator_real_loss=1.237, discriminator_fake_loss=1.193, generator_loss=29.85, generator_mel_loss=19.97, generator_kl_loss=1.945, generator_dur_loss=1.499, generator_adv_loss=2.41, generator_feat_match_loss=4.018, over 2517.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:24:05,791 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 04:24:14,153 INFO [train.py:534] (2/4) Epoch 319, validation: discriminator_loss=2.514, discriminator_real_loss=0.9557, discriminator_fake_loss=1.558, generator_loss=30.1, generator_mel_loss=20.71, generator_kl_loss=2.031, generator_dur_loss=1.489, generator_adv_loss=1.866, generator_feat_match_loss=4.004, over 100.00 samples.
2024-02-23 04:24:14,154 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 04:24:23,339 INFO [train.py:845] (2/4) Start epoch 320
2024-02-23 04:27:56,725 INFO [train.py:845] (2/4) Start epoch 321
2024-02-23 04:29:00,049 INFO [train.py:471] (2/4) Epoch 321, batch 10, global_batch_idx: 11850, batch size: 50, loss[discriminator_loss=2.516, discriminator_real_loss=1.34, discriminator_fake_loss=1.176, generator_loss=29.87, generator_mel_loss=19.79, generator_kl_loss=1.993, generator_dur_loss=1.521, generator_adv_loss=2.404, generator_feat_match_loss=4.164, over 50.00 samples.], tot_loss[discriminator_loss=2.351, discriminator_real_loss=1.145, discriminator_fake_loss=1.206, generator_loss=30.26, generator_mel_loss=20.04, generator_kl_loss=1.976, generator_dur_loss=1.501, generator_adv_loss=2.419, generator_feat_match_loss=4.317, over 815.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:31:23,650 INFO [train.py:845] (2/4) Start epoch 322
2024-02-23 04:33:37,193 INFO [train.py:471] (2/4) Epoch 322, batch 23, global_batch_idx: 11900, batch size: 55, loss[discriminator_loss=2.25, discriminator_real_loss=1.1, discriminator_fake_loss=1.149, generator_loss=30.72, generator_mel_loss=19.9, generator_kl_loss=1.926, generator_dur_loss=1.528, generator_adv_loss=2.818, generator_feat_match_loss=4.547, over 55.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.227, discriminator_fake_loss=1.173, generator_loss=30.2, generator_mel_loss=20.01, generator_kl_loss=1.935, generator_dur_loss=1.507, generator_adv_loss=2.459, generator_feat_match_loss=4.289, over 1703.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:34:51,093 INFO [train.py:845] (2/4) Start epoch 323
2024-02-23 04:38:19,514 INFO [train.py:471] (2/4) Epoch 323, batch 36, global_batch_idx: 11950, batch size: 79, loss[discriminator_loss=2.402, discriminator_real_loss=1.221, discriminator_fake_loss=1.181, generator_loss=29.93, generator_mel_loss=19.84, generator_kl_loss=1.91, generator_dur_loss=1.511, generator_adv_loss=2.422, generator_feat_match_loss=4.246, over 79.00 samples.], tot_loss[discriminator_loss=2.42, discriminator_real_loss=1.236, discriminator_fake_loss=1.184, generator_loss=30.28, generator_mel_loss=20.02, generator_kl_loss=1.931, generator_dur_loss=1.497, generator_adv_loss=2.475, generator_feat_match_loss=4.358, over 2785.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:38:19,997 INFO [train.py:845] (2/4) Start epoch 324
2024-02-23 04:41:51,703 INFO [train.py:845] (2/4) Start epoch 325
2024-02-23 04:43:14,069 INFO [train.py:471] (2/4) Epoch 325, batch 12, global_batch_idx: 12000, batch size: 59, loss[discriminator_loss=2.75, discriminator_real_loss=1.457, discriminator_fake_loss=1.294, generator_loss=29.49, generator_mel_loss=20.83, generator_kl_loss=1.998, generator_dur_loss=1.518, generator_adv_loss=2.209, generator_feat_match_loss=2.936, over 59.00 samples.], tot_loss[discriminator_loss=2.707, discriminator_real_loss=1.379, discriminator_fake_loss=1.329, generator_loss=29.77, generator_mel_loss=20.83, generator_kl_loss=1.978, generator_dur_loss=1.504, generator_adv_loss=2.152, generator_feat_match_loss=3.3, over 957.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 32.0
2024-02-23 04:43:14,070 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 04:43:22,885 INFO [train.py:534] (2/4) Epoch 325, validation: discriminator_loss=2.592, discriminator_real_loss=1.371, discriminator_fake_loss=1.221, generator_loss=30.65, generator_mel_loss=21.59, generator_kl_loss=2.037, generator_dur_loss=1.493, generator_adv_loss=2.201, generator_feat_match_loss=3.336, over 100.00 samples.
2024-02-23 04:43:22,886 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 04:45:33,230 INFO [train.py:845] (2/4) Start epoch 326
2024-02-23 04:48:05,749 INFO [train.py:471] (2/4) Epoch 326, batch 25, global_batch_idx: 12050, batch size: 95, loss[discriminator_loss=2.246, discriminator_real_loss=1.163, discriminator_fake_loss=1.084, generator_loss=30.96, generator_mel_loss=20.34, generator_kl_loss=1.983, generator_dur_loss=1.475, generator_adv_loss=2.549, generator_feat_match_loss=4.613, over 95.00 samples.], tot_loss[discriminator_loss=2.443, discriminator_real_loss=1.229, discriminator_fake_loss=1.214, generator_loss=30.1, generator_mel_loss=20.26, generator_kl_loss=1.941, generator_dur_loss=1.501, generator_adv_loss=2.379, generator_feat_match_loss=4.023, over 2001.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:49:04,414 INFO [train.py:845] (2/4) Start epoch 327
2024-02-23 04:52:38,890 INFO [train.py:845] (2/4) Start epoch 328
2024-02-23 04:52:58,602 INFO [train.py:471] (2/4) Epoch 328, batch 1, global_batch_idx: 12100, batch size: 54, loss[discriminator_loss=2.465, discriminator_real_loss=1.26, discriminator_fake_loss=1.205, generator_loss=29.91, generator_mel_loss=19.77, generator_kl_loss=1.84, generator_dur_loss=1.502, generator_adv_loss=2.416, generator_feat_match_loss=4.383, over 54.00 samples.], tot_loss[discriminator_loss=2.495, discriminator_real_loss=1.349, discriminator_fake_loss=1.146, generator_loss=30.15, generator_mel_loss=19.98, generator_kl_loss=1.887, generator_dur_loss=1.512, generator_adv_loss=2.448, generator_feat_match_loss=4.321, over 106.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:56:07,354 INFO [train.py:845] (2/4) Start epoch 329
2024-02-23 04:57:31,841 INFO [train.py:471] (2/4) Epoch 329, batch 14, global_batch_idx: 12150, batch size: 69, loss[discriminator_loss=2.381, discriminator_real_loss=1.169, discriminator_fake_loss=1.212, generator_loss=29.34, generator_mel_loss=19.81, generator_kl_loss=1.923, generator_dur_loss=1.503, generator_adv_loss=2.307, generator_feat_match_loss=3.791, over 69.00 samples.], tot_loss[discriminator_loss=2.43, discriminator_real_loss=1.226, discriminator_fake_loss=1.204, generator_loss=29.61, generator_mel_loss=19.97, generator_kl_loss=1.942, generator_dur_loss=1.497, generator_adv_loss=2.348, generator_feat_match_loss=3.85, over 1089.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 04:59:29,622 INFO [train.py:845] (2/4) Start epoch 330
2024-02-23 05:02:11,445 INFO [train.py:471] (2/4) Epoch 330, batch 27, global_batch_idx: 12200, batch size: 67, loss[discriminator_loss=2.414, discriminator_real_loss=1.215, discriminator_fake_loss=1.198, generator_loss=30.35, generator_mel_loss=20.44, generator_kl_loss=1.937, generator_dur_loss=1.509, generator_adv_loss=2.393, generator_feat_match_loss=4.074, over 67.00 samples.], tot_loss[discriminator_loss=2.389, discriminator_real_loss=1.221, discriminator_fake_loss=1.167, generator_loss=30.37, generator_mel_loss=20.07, generator_kl_loss=1.926, generator_dur_loss=1.498, generator_adv_loss=2.493, generator_feat_match_loss=4.384, over 2104.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:02:11,446 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 05:02:19,790 INFO [train.py:534] (2/4) Epoch 330, validation: discriminator_loss=2.448, discriminator_real_loss=1.187, discriminator_fake_loss=1.261, generator_loss=30.16, generator_mel_loss=20.3, generator_kl_loss=2.144, generator_dur_loss=1.508, generator_adv_loss=2.25, generator_feat_match_loss=3.959, over 100.00 samples.
2024-02-23 05:02:19,791 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 05:03:05,445 INFO [train.py:845] (2/4) Start epoch 331
2024-02-23 05:06:29,959 INFO [train.py:845] (2/4) Start epoch 332
2024-02-23 05:06:57,347 INFO [train.py:471] (2/4) Epoch 332, batch 3, global_batch_idx: 12250, batch size: 49, loss[discriminator_loss=2.359, discriminator_real_loss=1.163, discriminator_fake_loss=1.195, generator_loss=30.04, generator_mel_loss=19.81, generator_kl_loss=1.992, generator_dur_loss=1.497, generator_adv_loss=2.416, generator_feat_match_loss=4.332, over 49.00 samples.], tot_loss[discriminator_loss=2.384, discriminator_real_loss=1.231, discriminator_fake_loss=1.153, generator_loss=30.38, generator_mel_loss=20.15, generator_kl_loss=1.954, generator_dur_loss=1.513, generator_adv_loss=2.45, generator_feat_match_loss=4.308, over 231.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:09:56,055 INFO [train.py:845] (2/4) Start epoch 333
2024-02-23 05:11:32,309 INFO [train.py:471] (2/4) Epoch 333, batch 16, global_batch_idx: 12300, batch size: 55, loss[discriminator_loss=2.91, discriminator_real_loss=1.448, discriminator_fake_loss=1.461, generator_loss=30.68, generator_mel_loss=19.74, generator_kl_loss=1.959, generator_dur_loss=1.54, generator_adv_loss=3.285, generator_feat_match_loss=4.16, over 55.00 samples.], tot_loss[discriminator_loss=2.408, discriminator_real_loss=1.229, discriminator_fake_loss=1.18, generator_loss=30.5, generator_mel_loss=20.03, generator_kl_loss=1.956, generator_dur_loss=1.508, generator_adv_loss=2.568, generator_feat_match_loss=4.442, over 1236.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:13:27,309 INFO [train.py:845] (2/4) Start epoch 334
2024-02-23 05:16:12,044 INFO [train.py:471] (2/4) Epoch 334, batch 29, global_batch_idx: 12350, batch size: 76, loss[discriminator_loss=2.65, discriminator_real_loss=1.264, discriminator_fake_loss=1.387, generator_loss=29.98, generator_mel_loss=20.07, generator_kl_loss=2.035, generator_dur_loss=1.486, generator_adv_loss=2.441, generator_feat_match_loss=3.945, over 76.00 samples.], tot_loss[discriminator_loss=2.442, discriminator_real_loss=1.241, discriminator_fake_loss=1.201, generator_loss=30.11, generator_mel_loss=20.02, generator_kl_loss=1.947, generator_dur_loss=1.502, generator_adv_loss=2.465, generator_feat_match_loss=4.178, over 2094.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:16:50,556 INFO [train.py:845] (2/4) Start epoch 335
2024-02-23 05:20:22,093 INFO [train.py:845] (2/4) Start epoch 336
2024-02-23 05:21:03,985 INFO [train.py:471] (2/4) Epoch 336, batch 5, global_batch_idx: 12400, batch size: 51, loss[discriminator_loss=2.305, discriminator_real_loss=1.123, discriminator_fake_loss=1.182, generator_loss=30.43, generator_mel_loss=20.08, generator_kl_loss=1.921, generator_dur_loss=1.496, generator_adv_loss=2.404, generator_feat_match_loss=4.531, over 51.00 samples.], tot_loss[discriminator_loss=2.333, discriminator_real_loss=1.168, discriminator_fake_loss=1.165, generator_loss=29.87, generator_mel_loss=19.63, generator_kl_loss=1.932, generator_dur_loss=1.483, generator_adv_loss=2.45, generator_feat_match_loss=4.374, over 433.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 32.0
2024-02-23 05:21:03,986 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 05:21:13,070 INFO [train.py:534] (2/4) Epoch 336, validation: discriminator_loss=2.295, discriminator_real_loss=1.055, discriminator_fake_loss=1.24, generator_loss=31.13, generator_mel_loss=20.88, generator_kl_loss=1.995, generator_dur_loss=1.493, generator_adv_loss=2.297, generator_feat_match_loss=4.464, over 100.00 samples.
2024-02-23 05:21:13,070 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 05:24:00,854 INFO [train.py:845] (2/4) Start epoch 337
2024-02-23 05:25:50,221 INFO [train.py:471] (2/4) Epoch 337, batch 18, global_batch_idx: 12450, batch size: 67, loss[discriminator_loss=2.336, discriminator_real_loss=1.171, discriminator_fake_loss=1.164, generator_loss=30.29, generator_mel_loss=19.68, generator_kl_loss=2.047, generator_dur_loss=1.52, generator_adv_loss=2.529, generator_feat_match_loss=4.516, over 67.00 samples.], tot_loss[discriminator_loss=2.356, discriminator_real_loss=1.199, discriminator_fake_loss=1.156, generator_loss=30.25, generator_mel_loss=19.95, generator_kl_loss=1.933, generator_dur_loss=1.504, generator_adv_loss=2.473, generator_feat_match_loss=4.392, over 1317.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 32.0
2024-02-23 05:27:27,236 INFO [train.py:845] (2/4) Start epoch 338
2024-02-23 05:30:24,520 INFO [train.py:471] (2/4) Epoch 338, batch 31, global_batch_idx: 12500, batch size: 55, loss[discriminator_loss=2.465, discriminator_real_loss=1.337, discriminator_fake_loss=1.129, generator_loss=28.69, generator_mel_loss=19.02, generator_kl_loss=1.895, generator_dur_loss=1.528, generator_adv_loss=2.307, generator_feat_match_loss=3.941, over 55.00 samples.], tot_loss[discriminator_loss=2.453, discriminator_real_loss=1.276, discriminator_fake_loss=1.177, generator_loss=30.45, generator_mel_loss=20.05, generator_kl_loss=1.937, generator_dur_loss=1.499, generator_adv_loss=2.539, generator_feat_match_loss=4.42, over 2253.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:30:55,829 INFO [train.py:845] (2/4) Start epoch 339
2024-02-23 05:34:24,515 INFO [train.py:845] (2/4) Start epoch 340
2024-02-23 05:35:19,488 INFO [train.py:471] (2/4) Epoch 340, batch 7, global_batch_idx: 12550, batch size: 110, loss[discriminator_loss=2.311, discriminator_real_loss=1.161, discriminator_fake_loss=1.149, generator_loss=30.69, generator_mel_loss=20.28, generator_kl_loss=1.977, generator_dur_loss=1.47, generator_adv_loss=2.43, generator_feat_match_loss=4.531, over 110.00 samples.], tot_loss[discriminator_loss=2.384, discriminator_real_loss=1.221, discriminator_fake_loss=1.162, generator_loss=30.1, generator_mel_loss=20.03, generator_kl_loss=1.912, generator_dur_loss=1.498, generator_adv_loss=2.44, generator_feat_match_loss=4.223, over 583.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:37:52,881 INFO [train.py:845] (2/4) Start epoch 341
2024-02-23 05:40:02,107 INFO [train.py:471] (2/4) Epoch 341, batch 20, global_batch_idx: 12600, batch size: 50, loss[discriminator_loss=2.342, discriminator_real_loss=1.127, discriminator_fake_loss=1.215, generator_loss=30.52, generator_mel_loss=20.16, generator_kl_loss=1.828, generator_dur_loss=1.509, generator_adv_loss=2.586, generator_feat_match_loss=4.445, over 50.00 samples.], tot_loss[discriminator_loss=2.449, discriminator_real_loss=1.243, discriminator_fake_loss=1.206, generator_loss=30.69, generator_mel_loss=19.97, generator_kl_loss=1.942, generator_dur_loss=1.494, generator_adv_loss=2.632, generator_feat_match_loss=4.654, over 1633.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:40:02,108 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 05:40:10,089 INFO [train.py:534] (2/4) Epoch 341, validation: discriminator_loss=2.626, discriminator_real_loss=1.247, discriminator_fake_loss=1.379, generator_loss=29.93, generator_mel_loss=20.65, generator_kl_loss=1.95, generator_dur_loss=1.49, generator_adv_loss=2.082, generator_feat_match_loss=3.759, over 100.00 samples.
2024-02-23 05:40:10,090 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 05:41:31,680 INFO [train.py:845] (2/4) Start epoch 342
2024-02-23 05:44:43,832 INFO [train.py:471] (2/4) Epoch 342, batch 33, global_batch_idx: 12650, batch size: 69, loss[discriminator_loss=2.344, discriminator_real_loss=1.194, discriminator_fake_loss=1.148, generator_loss=29.91, generator_mel_loss=19.7, generator_kl_loss=1.888, generator_dur_loss=1.495, generator_adv_loss=2.438, generator_feat_match_loss=4.391, over 69.00 samples.], tot_loss[discriminator_loss=2.404, discriminator_real_loss=1.211, discriminator_fake_loss=1.193, generator_loss=30.29, generator_mel_loss=20.11, generator_kl_loss=1.965, generator_dur_loss=1.499, generator_adv_loss=2.445, generator_feat_match_loss=4.271, over 2489.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:45:00,275 INFO [train.py:845] (2/4) Start epoch 343
2024-02-23 05:48:29,426 INFO [train.py:845] (2/4) Start epoch 344
2024-02-23 05:49:31,259 INFO [train.py:471] (2/4) Epoch 344, batch 9, global_batch_idx: 12700, batch size: 61, loss[discriminator_loss=2.547, discriminator_real_loss=1.363, discriminator_fake_loss=1.184, generator_loss=30.15, generator_mel_loss=19.84, generator_kl_loss=1.977, generator_dur_loss=1.511, generator_adv_loss=2.445, generator_feat_match_loss=4.371, over 61.00 samples.], tot_loss[discriminator_loss=2.444, discriminator_real_loss=1.279, discriminator_fake_loss=1.165, generator_loss=30.95, generator_mel_loss=20.12, generator_kl_loss=1.902, generator_dur_loss=1.5, generator_adv_loss=2.664, generator_feat_match_loss=4.767, over 701.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:52:00,215 INFO [train.py:845] (2/4) Start epoch 345
2024-02-23 05:54:07,568 INFO [train.py:471] (2/4) Epoch 345, batch 22, global_batch_idx: 12750, batch size: 69, loss[discriminator_loss=2.355, discriminator_real_loss=1.138, discriminator_fake_loss=1.218, generator_loss=29.18, generator_mel_loss=19.37, generator_kl_loss=1.868, generator_dur_loss=1.488, generator_adv_loss=2.348, generator_feat_match_loss=4.109, over 69.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.215, discriminator_fake_loss=1.185, generator_loss=30, generator_mel_loss=19.89, generator_kl_loss=1.942, generator_dur_loss=1.499, generator_adv_loss=2.438, generator_feat_match_loss=4.229, over 1659.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 16.0
2024-02-23 05:55:32,186 INFO [train.py:845] (2/4) Start epoch 346
2024-02-23 05:58:58,116 INFO [train.py:471] (2/4) Epoch 346, batch 35, global_batch_idx: 12800, batch size: 63, loss[discriminator_loss=2.23, discriminator_real_loss=1.089, discriminator_fake_loss=1.141, generator_loss=29.99, generator_mel_loss=19.51, generator_kl_loss=1.83, generator_dur_loss=1.499, generator_adv_loss=2.422, generator_feat_match_loss=4.727, over 63.00 samples.], tot_loss[discriminator_loss=2.462, discriminator_real_loss=1.248, discriminator_fake_loss=1.214, generator_loss=29.89, generator_mel_loss=20.09, generator_kl_loss=1.939, generator_dur_loss=1.497, generator_adv_loss=2.393, generator_feat_match_loss=3.97, over 2622.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 32.0
2024-02-23 05:58:58,118 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 05:59:06,635 INFO [train.py:534] (2/4) Epoch 346, validation: discriminator_loss=2.175, discriminator_real_loss=0.9795, discriminator_fake_loss=1.196, generator_loss=31.71, generator_mel_loss=20.91, generator_kl_loss=1.973, generator_dur_loss=1.493, generator_adv_loss=2.358, generator_feat_match_loss=4.973, over 100.00 samples.
2024-02-23 05:59:06,636 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 05:59:11,534 INFO [train.py:845] (2/4) Start epoch 347
2024-02-23 06:02:37,050 INFO [train.py:845] (2/4) Start epoch 348
2024-02-23 06:03:47,305 INFO [train.py:471] (2/4) Epoch 348, batch 11, global_batch_idx: 12850, batch size: 76, loss[discriminator_loss=2.43, discriminator_real_loss=1.248, discriminator_fake_loss=1.182, generator_loss=30.07, generator_mel_loss=20.15, generator_kl_loss=1.988, generator_dur_loss=1.517, generator_adv_loss=2.387, generator_feat_match_loss=4.035, over 76.00 samples.], tot_loss[discriminator_loss=2.416, discriminator_real_loss=1.218, discriminator_fake_loss=1.198, generator_loss=30.11, generator_mel_loss=20.07, generator_kl_loss=1.934, generator_dur_loss=1.5, generator_adv_loss=2.42, generator_feat_match_loss=4.19, over 865.00 samples.], cur_lr_g: 1.92e-04, cur_lr_d: 1.92e-04, grad_scale: 32.0
2024-02-23 06:06:01,463 INFO [train.py:845] (2/4) Start epoch 349
2024-02-23 06:08:25,291 INFO [train.py:471] (2/4) Epoch 349, batch 24, global_batch_idx: 12900, batch size: 56, loss[discriminator_loss=2.434, discriminator_real_loss=1.244, discriminator_fake_loss=1.188, generator_loss=29.61, generator_mel_loss=19.93, generator_kl_loss=1.96, generator_dur_loss=1.494, generator_adv_loss=2.33, generator_feat_match_loss=3.896, over 56.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.205, discriminator_fake_loss=1.197, generator_loss=29.86, generator_mel_loss=19.9, generator_kl_loss=1.906, generator_dur_loss=1.502, generator_adv_loss=2.405, generator_feat_match_loss=4.147, over 1773.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 32.0
2024-02-23 06:09:24,820 INFO [train.py:845] (2/4) Start epoch 350
2024-02-23 06:12:54,569 INFO [train.py:845] (2/4) Start epoch 351
2024-02-23 06:13:08,146 INFO [train.py:471] (2/4) Epoch 351, batch 0, global_batch_idx: 12950, batch size: 52, loss[discriminator_loss=2.512, discriminator_real_loss=1.25, discriminator_fake_loss=1.261, generator_loss=28.83, generator_mel_loss=19.37, generator_kl_loss=1.904, generator_dur_loss=1.503, generator_adv_loss=2.322, generator_feat_match_loss=3.725, over 52.00 samples.], tot_loss[discriminator_loss=2.512, discriminator_real_loss=1.25, discriminator_fake_loss=1.261, generator_loss=28.83, generator_mel_loss=19.37, generator_kl_loss=1.904, generator_dur_loss=1.503, generator_adv_loss=2.322, generator_feat_match_loss=3.725, over 52.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 06:16:24,415 INFO [train.py:845] (2/4) Start epoch 352
2024-02-23 06:17:38,363 INFO [train.py:471] (2/4) Epoch 352, batch 13, global_batch_idx: 13000, batch size: 58, loss[discriminator_loss=2.152, discriminator_real_loss=1.145, discriminator_fake_loss=1.007, generator_loss=30.62, generator_mel_loss=19.84, generator_kl_loss=1.978, generator_dur_loss=1.501, generator_adv_loss=2.568, generator_feat_match_loss=4.738, over 58.00 samples.], tot_loss[discriminator_loss=2.393, discriminator_real_loss=1.228, discriminator_fake_loss=1.165, generator_loss=30.49, generator_mel_loss=19.9, generator_kl_loss=1.94, generator_dur_loss=1.498, generator_adv_loss=2.527, generator_feat_match_loss=4.622, over 1028.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 06:17:38,364 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 06:17:46,855 INFO [train.py:534] (2/4) Epoch 352, validation: discriminator_loss=2.179, discriminator_real_loss=0.9878, discriminator_fake_loss=1.191, generator_loss=32.01, generator_mel_loss=21.16, generator_kl_loss=2, generator_dur_loss=1.499, generator_adv_loss=2.444, generator_feat_match_loss=4.908, over 100.00 samples.
2024-02-23 06:17:46,856 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 06:19:58,655 INFO [train.py:845] (2/4) Start epoch 353
2024-02-23 06:22:35,587 INFO [train.py:471] (2/4) Epoch 353, batch 26, global_batch_idx: 13050, batch size: 60, loss[discriminator_loss=2.771, discriminator_real_loss=1.605, discriminator_fake_loss=1.166, generator_loss=29.55, generator_mel_loss=19.53, generator_kl_loss=1.952, generator_dur_loss=1.487, generator_adv_loss=2.443, generator_feat_match_loss=4.137, over 60.00 samples.], tot_loss[discriminator_loss=2.436, discriminator_real_loss=1.26, discriminator_fake_loss=1.176, generator_loss=30.61, generator_mel_loss=19.97, generator_kl_loss=1.941, generator_dur_loss=1.498, generator_adv_loss=2.587, generator_feat_match_loss=4.613, over 2095.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 06:23:29,159 INFO [train.py:845] (2/4) Start epoch 354
2024-02-23 06:26:57,656 INFO [train.py:845] (2/4) Start epoch 355
2024-02-23 06:27:25,317 INFO [train.py:471] (2/4) Epoch 355, batch 2, global_batch_idx: 13100, batch size: 154, loss[discriminator_loss=2.48, discriminator_real_loss=1.194, discriminator_fake_loss=1.285, generator_loss=30.66, generator_mel_loss=20.35, generator_kl_loss=1.98, generator_dur_loss=1.473, generator_adv_loss=2.352, generator_feat_match_loss=4.504, over 154.00 samples.], tot_loss[discriminator_loss=2.558, discriminator_real_loss=1.234, discriminator_fake_loss=1.324, generator_loss=30.43, generator_mel_loss=20.3, generator_kl_loss=1.951, generator_dur_loss=1.484, generator_adv_loss=2.298, generator_feat_match_loss=4.4, over 317.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 06:30:23,505 INFO [train.py:845] (2/4) Start epoch 356
2024-02-23 06:31:59,785 INFO [train.py:471] (2/4) Epoch 356, batch 15, global_batch_idx: 13150, batch size: 56, loss[discriminator_loss=2.535, discriminator_real_loss=1.336, discriminator_fake_loss=1.198, generator_loss=30.61, generator_mel_loss=20.95, generator_kl_loss=1.998, generator_dur_loss=1.494, generator_adv_loss=2.463, generator_feat_match_loss=3.709, over 56.00 samples.], tot_loss[discriminator_loss=2.427, discriminator_real_loss=1.18, discriminator_fake_loss=1.246, generator_loss=29.67, generator_mel_loss=19.86, generator_kl_loss=1.937, generator_dur_loss=1.492, generator_adv_loss=2.331, generator_feat_match_loss=4.052, over 1169.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 06:33:50,081 INFO [train.py:845] (2/4) Start epoch 357
2024-02-23 06:36:27,713 INFO [train.py:471] (2/4) Epoch 357, batch 28, global_batch_idx: 13200, batch size: 95, loss[discriminator_loss=2.564, discriminator_real_loss=1.19, discriminator_fake_loss=1.374, generator_loss=29.79, generator_mel_loss=20.09, generator_kl_loss=1.964, generator_dur_loss=1.49, generator_adv_loss=2.285, generator_feat_match_loss=3.959, over 95.00 samples.], tot_loss[discriminator_loss=2.386, discriminator_real_loss=1.21, discriminator_fake_loss=1.176, generator_loss=30.49, generator_mel_loss=19.98, generator_kl_loss=1.951, generator_dur_loss=1.499, generator_adv_loss=2.507, generator_feat_match_loss=4.558, over 2096.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 32.0
2024-02-23 06:36:27,714 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 06:36:36,455 INFO [train.py:534] (2/4) Epoch 357, validation: discriminator_loss=2.469, discriminator_real_loss=1.151, discriminator_fake_loss=1.318, generator_loss=30.55, generator_mel_loss=20.84, generator_kl_loss=2.001, generator_dur_loss=1.496, generator_adv_loss=2.216, generator_feat_match_loss=4, over 100.00 samples.
2024-02-23 06:36:36,455 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 06:37:18,061 INFO [train.py:845] (2/4) Start epoch 358
2024-02-23 06:40:48,421 INFO [train.py:845] (2/4) Start epoch 359
2024-02-23 06:41:24,865 INFO [train.py:471] (2/4) Epoch 359, batch 4, global_batch_idx: 13250, batch size: 71, loss[discriminator_loss=2.375, discriminator_real_loss=1.303, discriminator_fake_loss=1.071, generator_loss=30.47, generator_mel_loss=20.01, generator_kl_loss=1.929, generator_dur_loss=1.486, generator_adv_loss=2.531, generator_feat_match_loss=4.516, over 71.00 samples.], tot_loss[discriminator_loss=2.585, discriminator_real_loss=1.353, discriminator_fake_loss=1.232, generator_loss=30.64, generator_mel_loss=20.31, generator_kl_loss=1.901, generator_dur_loss=1.501, generator_adv_loss=2.44, generator_feat_match_loss=4.494, over 442.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 32.0
2024-02-23 06:44:08,300 INFO [train.py:845] (2/4) Start epoch 360
2024-02-23 06:46:07,023 INFO [train.py:471] (2/4) Epoch 360, batch 17, global_batch_idx: 13300, batch size: 154, loss[discriminator_loss=2.291, discriminator_real_loss=1.157, discriminator_fake_loss=1.134, generator_loss=30.3, generator_mel_loss=20.01, generator_kl_loss=1.915, generator_dur_loss=1.465, generator_adv_loss=2.463, generator_feat_match_loss=4.449, over 154.00 samples.], tot_loss[discriminator_loss=2.408, discriminator_real_loss=1.192, discriminator_fake_loss=1.216, generator_loss=29.65, generator_mel_loss=19.8, generator_kl_loss=1.939, generator_dur_loss=1.493, generator_adv_loss=2.377, generator_feat_match_loss=4.037, over 1287.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 32.0
2024-02-23 06:47:43,193 INFO [train.py:845] (2/4) Start epoch 361
2024-02-23 06:50:45,482 INFO [train.py:471] (2/4) Epoch 361, batch 30, global_batch_idx: 13350, batch size: 52, loss[discriminator_loss=2.855, discriminator_real_loss=1.54, discriminator_fake_loss=1.315, generator_loss=27.77, generator_mel_loss=19.69, generator_kl_loss=1.962, generator_dur_loss=1.517, generator_adv_loss=1.867, generator_feat_match_loss=2.742, over 52.00 samples.], tot_loss[discriminator_loss=3.001, discriminator_real_loss=1.635, discriminator_fake_loss=1.366, generator_loss=29.02, generator_mel_loss=20.12, generator_kl_loss=1.963, generator_dur_loss=1.494, generator_adv_loss=2.235, generator_feat_match_loss=3.212, over 2450.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 32.0
2024-02-23 06:51:17,624 INFO [train.py:845] (2/4) Start epoch 362
2024-02-23 06:54:48,623 INFO [train.py:845] (2/4) Start epoch 363
2024-02-23 06:55:26,514 INFO [train.py:471] (2/4) Epoch 363, batch 6, global_batch_idx: 13400, batch size: 81, loss[discriminator_loss=2.57, discriminator_real_loss=1.372, discriminator_fake_loss=1.198, generator_loss=29.6, generator_mel_loss=20.58, generator_kl_loss=1.899, generator_dur_loss=1.516, generator_adv_loss=2.088, generator_feat_match_loss=3.508, over 81.00 samples.], tot_loss[discriminator_loss=2.585, discriminator_real_loss=1.304, discriminator_fake_loss=1.281, generator_loss=29.36, generator_mel_loss=20.38, generator_kl_loss=1.9, generator_dur_loss=1.496, generator_adv_loss=2.16, generator_feat_match_loss=3.417, over 459.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 06:55:26,515 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 06:55:35,274 INFO [train.py:534] (2/4) Epoch 363, validation: discriminator_loss=2.665, discriminator_real_loss=1.161, discriminator_fake_loss=1.504, generator_loss=29.77, generator_mel_loss=21.12, generator_kl_loss=2.113, generator_dur_loss=1.49, generator_adv_loss=1.811, generator_feat_match_loss=3.239, over 100.00 samples.
2024-02-23 06:55:35,275 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 06:58:26,450 INFO [train.py:845] (2/4) Start epoch 364
2024-02-23 07:00:27,293 INFO [train.py:471] (2/4) Epoch 364, batch 19, global_batch_idx: 13450, batch size: 81, loss[discriminator_loss=2.752, discriminator_real_loss=1.097, discriminator_fake_loss=1.655, generator_loss=29.14, generator_mel_loss=20.33, generator_kl_loss=1.972, generator_dur_loss=1.471, generator_adv_loss=2.084, generator_feat_match_loss=3.279, over 81.00 samples.], tot_loss[discriminator_loss=2.652, discriminator_real_loss=1.341, discriminator_fake_loss=1.31, generator_loss=29.48, generator_mel_loss=20.28, generator_kl_loss=1.92, generator_dur_loss=1.497, generator_adv_loss=2.253, generator_feat_match_loss=3.529, over 1338.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:01:58,711 INFO [train.py:845] (2/4) Start epoch 365
2024-02-23 07:05:02,223 INFO [train.py:471] (2/4) Epoch 365, batch 32, global_batch_idx: 13500, batch size: 50, loss[discriminator_loss=2.832, discriminator_real_loss=1.27, discriminator_fake_loss=1.563, generator_loss=29.73, generator_mel_loss=21.01, generator_kl_loss=1.989, generator_dur_loss=1.498, generator_adv_loss=1.938, generator_feat_match_loss=3.297, over 50.00 samples.], tot_loss[discriminator_loss=2.514, discriminator_real_loss=1.253, discriminator_fake_loss=1.26, generator_loss=29.91, generator_mel_loss=20.42, generator_kl_loss=1.915, generator_dur_loss=1.501, generator_adv_loss=2.274, generator_feat_match_loss=3.802, over 2282.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:05:24,324 INFO [train.py:845] (2/4) Start epoch 366
2024-02-23 07:08:53,020 INFO [train.py:845] (2/4) Start epoch 367
2024-02-23 07:09:50,975 INFO [train.py:471] (2/4) Epoch 367, batch 8, global_batch_idx: 13550, batch size: 51, loss[discriminator_loss=2.588, discriminator_real_loss=1.209, discriminator_fake_loss=1.379, generator_loss=29.89, generator_mel_loss=20.66, generator_kl_loss=2.043, generator_dur_loss=1.505, generator_adv_loss=2.119, generator_feat_match_loss=3.568, over 51.00 samples.], tot_loss[discriminator_loss=2.425, discriminator_real_loss=1.213, discriminator_fake_loss=1.212, generator_loss=30.03, generator_mel_loss=20.03, generator_kl_loss=1.967, generator_dur_loss=1.508, generator_adv_loss=2.373, generator_feat_match_loss=4.158, over 554.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 8.0
2024-02-23 07:12:20,095 INFO [train.py:845] (2/4) Start epoch 368
2024-02-23 07:14:36,909 INFO [train.py:471] (2/4) Epoch 368, batch 21, global_batch_idx: 13600, batch size: 60, loss[discriminator_loss=2.441, discriminator_real_loss=1.331, discriminator_fake_loss=1.109, generator_loss=29.23, generator_mel_loss=20.01, generator_kl_loss=1.98, generator_dur_loss=1.495, generator_adv_loss=2.096, generator_feat_match_loss=3.65, over 60.00 samples.], tot_loss[discriminator_loss=2.693, discriminator_real_loss=1.388, discriminator_fake_loss=1.305, generator_loss=29.15, generator_mel_loss=20.09, generator_kl_loss=1.937, generator_dur_loss=1.499, generator_adv_loss=2.149, generator_feat_match_loss=3.472, over 1697.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:14:36,910 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 07:14:45,797 INFO [train.py:534] (2/4) Epoch 368, validation: discriminator_loss=2.368, discriminator_real_loss=1.078, discriminator_fake_loss=1.29, generator_loss=30.5, generator_mel_loss=20.97, generator_kl_loss=2.089, generator_dur_loss=1.49, generator_adv_loss=1.956, generator_feat_match_loss=3.988, over 100.00 samples.
2024-02-23 07:14:45,798 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 07:16:03,218 INFO [train.py:845] (2/4) Start epoch 369
2024-02-23 07:19:16,951 INFO [train.py:471] (2/4) Epoch 369, batch 34, global_batch_idx: 13650, batch size: 56, loss[discriminator_loss=2.254, discriminator_real_loss=1.063, discriminator_fake_loss=1.189, generator_loss=30.59, generator_mel_loss=20.23, generator_kl_loss=2.031, generator_dur_loss=1.509, generator_adv_loss=2.215, generator_feat_match_loss=4.598, over 56.00 samples.], tot_loss[discriminator_loss=2.43, discriminator_real_loss=1.242, discriminator_fake_loss=1.188, generator_loss=29.92, generator_mel_loss=19.9, generator_kl_loss=1.922, generator_dur_loss=1.502, generator_adv_loss=2.44, generator_feat_match_loss=4.153, over 2175.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:19:30,886 INFO [train.py:845] (2/4) Start epoch 370
2024-02-23 07:23:01,335 INFO [train.py:845] (2/4) Start epoch 371
2024-02-23 07:24:08,307 INFO [train.py:471] (2/4) Epoch 371, batch 10, global_batch_idx: 13700, batch size: 56, loss[discriminator_loss=2.691, discriminator_real_loss=1.443, discriminator_fake_loss=1.248, generator_loss=29.66, generator_mel_loss=19.65, generator_kl_loss=2.012, generator_dur_loss=1.493, generator_adv_loss=2.619, generator_feat_match_loss=3.881, over 56.00 samples.], tot_loss[discriminator_loss=2.307, discriminator_real_loss=1.093, discriminator_fake_loss=1.214, generator_loss=30.51, generator_mel_loss=20.04, generator_kl_loss=1.976, generator_dur_loss=1.498, generator_adv_loss=2.466, generator_feat_match_loss=4.524, over 705.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:26:27,793 INFO [train.py:845] (2/4) Start epoch 372
2024-02-23 07:28:45,188 INFO [train.py:471] (2/4) Epoch 372, batch 23, global_batch_idx: 13750, batch size: 55, loss[discriminator_loss=2.449, discriminator_real_loss=1.247, discriminator_fake_loss=1.202, generator_loss=30.43, generator_mel_loss=20.25, generator_kl_loss=1.942, generator_dur_loss=1.51, generator_adv_loss=2.498, generator_feat_match_loss=4.227, over 55.00 samples.], tot_loss[discriminator_loss=2.427, discriminator_real_loss=1.203, discriminator_fake_loss=1.224, generator_loss=29.83, generator_mel_loss=19.88, generator_kl_loss=1.929, generator_dur_loss=1.502, generator_adv_loss=2.369, generator_feat_match_loss=4.15, over 1514.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:29:55,559 INFO [train.py:845] (2/4) Start epoch 373
2024-02-23 07:33:23,721 INFO [train.py:471] (2/4) Epoch 373, batch 36, global_batch_idx: 13800, batch size: 64, loss[discriminator_loss=2.277, discriminator_real_loss=1.15, discriminator_fake_loss=1.127, generator_loss=30.29, generator_mel_loss=19.73, generator_kl_loss=1.921, generator_dur_loss=1.509, generator_adv_loss=2.436, generator_feat_match_loss=4.695, over 64.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.206, discriminator_fake_loss=1.195, generator_loss=30.22, generator_mel_loss=19.99, generator_kl_loss=1.936, generator_dur_loss=1.494, generator_adv_loss=2.425, generator_feat_match_loss=4.375, over 2690.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:33:23,723 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 07:33:32,093 INFO [train.py:534] (2/4) Epoch 373, validation: discriminator_loss=2.189, discriminator_real_loss=0.9941, discriminator_fake_loss=1.195, generator_loss=31.43, generator_mel_loss=20.77, generator_kl_loss=1.955, generator_dur_loss=1.489, generator_adv_loss=2.378, generator_feat_match_loss=4.845, over 100.00 samples.
2024-02-23 07:33:32,094 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 07:33:32,459 INFO [train.py:845] (2/4) Start epoch 374
2024-02-23 07:36:55,121 INFO [train.py:845] (2/4) Start epoch 375
2024-02-23 07:38:05,468 INFO [train.py:471] (2/4) Epoch 375, batch 12, global_batch_idx: 13850, batch size: 59, loss[discriminator_loss=2.41, discriminator_real_loss=1.305, discriminator_fake_loss=1.105, generator_loss=30.06, generator_mel_loss=20.01, generator_kl_loss=1.965, generator_dur_loss=1.493, generator_adv_loss=2.453, generator_feat_match_loss=4.133, over 59.00 samples.], tot_loss[discriminator_loss=2.403, discriminator_real_loss=1.212, discriminator_fake_loss=1.191, generator_loss=29.83, generator_mel_loss=19.78, generator_kl_loss=1.908, generator_dur_loss=1.498, generator_adv_loss=2.426, generator_feat_match_loss=4.222, over 840.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:40:20,470 INFO [train.py:845] (2/4) Start epoch 376
2024-02-23 07:42:44,679 INFO [train.py:471] (2/4) Epoch 376, batch 25, global_batch_idx: 13900, batch size: 54, loss[discriminator_loss=2.879, discriminator_real_loss=1.648, discriminator_fake_loss=1.23, generator_loss=30.22, generator_mel_loss=20.08, generator_kl_loss=1.942, generator_dur_loss=1.508, generator_adv_loss=2.555, generator_feat_match_loss=4.129, over 54.00 samples.], tot_loss[discriminator_loss=2.436, discriminator_real_loss=1.238, discriminator_fake_loss=1.198, generator_loss=30.42, generator_mel_loss=20.05, generator_kl_loss=1.936, generator_dur_loss=1.497, generator_adv_loss=2.511, generator_feat_match_loss=4.426, over 1725.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:43:44,228 INFO [train.py:845] (2/4) Start epoch 377
2024-02-23 07:47:11,822 INFO [train.py:845] (2/4) Start epoch 378
2024-02-23 07:47:31,311 INFO [train.py:471] (2/4) Epoch 378, batch 1, global_batch_idx: 13950, batch size: 52, loss[discriminator_loss=2.488, discriminator_real_loss=1.322, discriminator_fake_loss=1.166, generator_loss=29.14, generator_mel_loss=19.71, generator_kl_loss=2.064, generator_dur_loss=1.482, generator_adv_loss=2.422, generator_feat_match_loss=3.465, over 52.00 samples.], tot_loss[discriminator_loss=2.526, discriminator_real_loss=1.301, discriminator_fake_loss=1.225, generator_loss=29.3, generator_mel_loss=19.68, generator_kl_loss=1.967, generator_dur_loss=1.499, generator_adv_loss=2.406, generator_feat_match_loss=3.751, over 153.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:50:47,050 INFO [train.py:845] (2/4) Start epoch 379
2024-02-23 07:52:12,011 INFO [train.py:471] (2/4) Epoch 379, batch 14, global_batch_idx: 14000, batch size: 101, loss[discriminator_loss=2.219, discriminator_real_loss=1.045, discriminator_fake_loss=1.174, generator_loss=30.94, generator_mel_loss=20.22, generator_kl_loss=1.996, generator_dur_loss=1.479, generator_adv_loss=2.404, generator_feat_match_loss=4.84, over 101.00 samples.], tot_loss[discriminator_loss=2.457, discriminator_real_loss=1.226, discriminator_fake_loss=1.231, generator_loss=29.87, generator_mel_loss=19.97, generator_kl_loss=1.943, generator_dur_loss=1.499, generator_adv_loss=2.38, generator_feat_match_loss=4.08, over 963.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 32.0
2024-02-23 07:52:12,013 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 07:52:20,427 INFO [train.py:534] (2/4) Epoch 379, validation: discriminator_loss=2.154, discriminator_real_loss=0.9438, discriminator_fake_loss=1.21, generator_loss=31.7, generator_mel_loss=20.77, generator_kl_loss=2.049, generator_dur_loss=1.48, generator_adv_loss=2.359, generator_feat_match_loss=5.046, over 100.00 samples.
2024-02-23 07:52:20,427 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 07:54:21,522 INFO [train.py:845] (2/4) Start epoch 380
2024-02-23 07:56:50,601 INFO [train.py:471] (2/4) Epoch 380, batch 27, global_batch_idx: 14050, batch size: 61, loss[discriminator_loss=2.773, discriminator_real_loss=1.598, discriminator_fake_loss=1.175, generator_loss=29.33, generator_mel_loss=19.71, generator_kl_loss=1.994, generator_dur_loss=1.504, generator_adv_loss=2.545, generator_feat_match_loss=3.576, over 61.00 samples.], tot_loss[discriminator_loss=2.523, discriminator_real_loss=1.248, discriminator_fake_loss=1.275, generator_loss=29.83, generator_mel_loss=20.16, generator_kl_loss=1.928, generator_dur_loss=1.494, generator_adv_loss=2.33, generator_feat_match_loss=3.925, over 2072.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 07:57:41,837 INFO [train.py:845] (2/4) Start epoch 381
2024-02-23 08:01:08,148 INFO [train.py:845] (2/4) Start epoch 382
2024-02-23 08:01:37,741 INFO [train.py:471] (2/4) Epoch 382, batch 3, global_batch_idx: 14100, batch size: 59, loss[discriminator_loss=2.742, discriminator_real_loss=1.493, discriminator_fake_loss=1.248, generator_loss=30.17, generator_mel_loss=20.29, generator_kl_loss=1.938, generator_dur_loss=1.514, generator_adv_loss=2.5, generator_feat_match_loss=3.926, over 59.00 samples.], tot_loss[discriminator_loss=2.565, discriminator_real_loss=1.289, discriminator_fake_loss=1.276, generator_loss=30.04, generator_mel_loss=20.41, generator_kl_loss=1.97, generator_dur_loss=1.492, generator_adv_loss=2.283, generator_feat_match_loss=3.885, over 270.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 8.0
2024-02-23 08:04:36,318 INFO [train.py:845] (2/4) Start epoch 383
2024-02-23 08:06:21,078 INFO [train.py:471] (2/4) Epoch 383, batch 16, global_batch_idx: 14150, batch size: 79, loss[discriminator_loss=2.529, discriminator_real_loss=1.361, discriminator_fake_loss=1.168, generator_loss=29.16, generator_mel_loss=19.43, generator_kl_loss=1.885, generator_dur_loss=1.513, generator_adv_loss=2.465, generator_feat_match_loss=3.867, over 79.00 samples.], tot_loss[discriminator_loss=2.494, discriminator_real_loss=1.253, discriminator_fake_loss=1.241, generator_loss=29.54, generator_mel_loss=19.76, generator_kl_loss=1.934, generator_dur_loss=1.493, generator_adv_loss=2.338, generator_feat_match_loss=4.016, over 1305.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 8.0
2024-02-23 08:08:09,091 INFO [train.py:845] (2/4) Start epoch 384
2024-02-23 08:11:02,653 INFO [train.py:471] (2/4) Epoch 384, batch 29, global_batch_idx: 14200, batch size: 63, loss[discriminator_loss=2.328, discriminator_real_loss=1.217, discriminator_fake_loss=1.11, generator_loss=29.83, generator_mel_loss=19.67, generator_kl_loss=1.871, generator_dur_loss=1.485, generator_adv_loss=2.439, generator_feat_match_loss=4.355, over 63.00 samples.], tot_loss[discriminator_loss=2.436, discriminator_real_loss=1.223, discriminator_fake_loss=1.213, generator_loss=30, generator_mel_loss=20.04, generator_kl_loss=1.943, generator_dur_loss=1.497, generator_adv_loss=2.381, generator_feat_match_loss=4.149, over 2110.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 8.0
2024-02-23 08:11:02,654 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 08:11:10,778 INFO [train.py:534] (2/4) Epoch 384, validation: discriminator_loss=2.272, discriminator_real_loss=1.044, discriminator_fake_loss=1.228, generator_loss=30.95, generator_mel_loss=20.51, generator_kl_loss=1.974, generator_dur_loss=1.491, generator_adv_loss=2.349, generator_feat_match_loss=4.633, over 100.00 samples.
2024-02-23 08:11:10,779 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 08:11:45,331 INFO [train.py:845] (2/4) Start epoch 385
2024-02-23 08:15:14,514 INFO [train.py:845] (2/4) Start epoch 386
2024-02-23 08:15:54,561 INFO [train.py:471] (2/4) Epoch 386, batch 5, global_batch_idx: 14250, batch size: 52, loss[discriminator_loss=2.236, discriminator_real_loss=1.189, discriminator_fake_loss=1.047, generator_loss=30.14, generator_mel_loss=19.15, generator_kl_loss=1.966, generator_dur_loss=1.503, generator_adv_loss=2.494, generator_feat_match_loss=5.027, over 52.00 samples.], tot_loss[discriminator_loss=2.305, discriminator_real_loss=1.218, discriminator_fake_loss=1.087, generator_loss=30.47, generator_mel_loss=19.78, generator_kl_loss=1.908, generator_dur_loss=1.496, generator_adv_loss=2.553, generator_feat_match_loss=4.734, over 413.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 8.0
2024-02-23 08:18:45,057 INFO [train.py:845] (2/4) Start epoch 387
2024-02-23 08:20:36,761 INFO [train.py:471] (2/4) Epoch 387, batch 18, global_batch_idx: 14300, batch size: 69, loss[discriminator_loss=2.408, discriminator_real_loss=1.182, discriminator_fake_loss=1.227, generator_loss=28.7, generator_mel_loss=18.91, generator_kl_loss=1.924, generator_dur_loss=1.474, generator_adv_loss=2.418, generator_feat_match_loss=3.973, over 69.00 samples.], tot_loss[discriminator_loss=2.408, discriminator_real_loss=1.217, discriminator_fake_loss=1.191, generator_loss=29.69, generator_mel_loss=19.61, generator_kl_loss=1.947, generator_dur_loss=1.494, generator_adv_loss=2.419, generator_feat_match_loss=4.219, over 1426.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 8.0
2024-02-23 08:22:14,626 INFO [train.py:845] (2/4) Start epoch 388
2024-02-23 08:25:21,577 INFO [train.py:471] (2/4) Epoch 388, batch 31, global_batch_idx: 14350, batch size: 53, loss[discriminator_loss=2.354, discriminator_real_loss=1.152, discriminator_fake_loss=1.201, generator_loss=30.2, generator_mel_loss=19.8, generator_kl_loss=1.992, generator_dur_loss=1.506, generator_adv_loss=2.504, generator_feat_match_loss=4.398, over 53.00 samples.], tot_loss[discriminator_loss=2.364, discriminator_real_loss=1.198, discriminator_fake_loss=1.167, generator_loss=30.3, generator_mel_loss=19.81, generator_kl_loss=1.951, generator_dur_loss=1.492, generator_adv_loss=2.475, generator_feat_match_loss=4.575, over 2535.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 8.0
2024-02-23 08:25:43,582 INFO [train.py:845] (2/4) Start epoch 389
2024-02-23 08:29:12,057 INFO [train.py:845] (2/4) Start epoch 390
2024-02-23 08:29:59,954 INFO [train.py:471] (2/4) Epoch 390, batch 7, global_batch_idx: 14400, batch size: 53, loss[discriminator_loss=2.57, discriminator_real_loss=1.42, discriminator_fake_loss=1.149, generator_loss=28.97, generator_mel_loss=19.53, generator_kl_loss=1.819, generator_dur_loss=1.518, generator_adv_loss=2.439, generator_feat_match_loss=3.664, over 53.00 samples.], tot_loss[discriminator_loss=2.546, discriminator_real_loss=1.33, discriminator_fake_loss=1.216, generator_loss=29.26, generator_mel_loss=19.66, generator_kl_loss=1.91, generator_dur_loss=1.508, generator_adv_loss=2.443, generator_feat_match_loss=3.737, over 543.00 samples.], cur_lr_g: 1.91e-04, cur_lr_d: 1.91e-04, grad_scale: 16.0
2024-02-23 08:29:59,956 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 08:30:09,078 INFO [train.py:534] (2/4) Epoch 390, validation: discriminator_loss=2.552, discriminator_real_loss=1.306, discriminator_fake_loss=1.247, generator_loss=30.48, generator_mel_loss=20.58, generator_kl_loss=2.092, generator_dur_loss=1.486, generator_adv_loss=2.381, generator_feat_match_loss=3.942, over 100.00 samples.
2024-02-23 08:30:09,079 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 08:32:50,901 INFO [train.py:845] (2/4) Start epoch 391
2024-02-23 08:34:51,550 INFO [train.py:471] (2/4) Epoch 391, batch 20, global_batch_idx: 14450, batch size: 65, loss[discriminator_loss=2.342, discriminator_real_loss=1.166, discriminator_fake_loss=1.176, generator_loss=29.69, generator_mel_loss=19.65, generator_kl_loss=1.904, generator_dur_loss=1.491, generator_adv_loss=2.363, generator_feat_match_loss=4.277, over 65.00 samples.], tot_loss[discriminator_loss=2.369, discriminator_real_loss=1.174, discriminator_fake_loss=1.195, generator_loss=29.7, generator_mel_loss=19.66, generator_kl_loss=1.953, generator_dur_loss=1.497, generator_adv_loss=2.382, generator_feat_match_loss=4.207, over 1571.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 08:36:20,010 INFO [train.py:845] (2/4) Start epoch 392
2024-02-23 08:39:25,232 INFO [train.py:471] (2/4) Epoch 392, batch 33, global_batch_idx: 14500, batch size: 73, loss[discriminator_loss=2.391, discriminator_real_loss=1.169, discriminator_fake_loss=1.223, generator_loss=30.32, generator_mel_loss=19.79, generator_kl_loss=1.894, generator_dur_loss=1.487, generator_adv_loss=2.395, generator_feat_match_loss=4.754, over 73.00 samples.], tot_loss[discriminator_loss=2.419, discriminator_real_loss=1.231, discriminator_fake_loss=1.187, generator_loss=30.39, generator_mel_loss=19.94, generator_kl_loss=1.942, generator_dur_loss=1.498, generator_adv_loss=2.476, generator_feat_match_loss=4.534, over 2270.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 08:39:46,829 INFO [train.py:845] (2/4) Start epoch 393
2024-02-23 08:43:08,513 INFO [train.py:845] (2/4) Start epoch 394
2024-02-23 08:44:14,894 INFO [train.py:471] (2/4) Epoch 394, batch 9, global_batch_idx: 14550, batch size: 52, loss[discriminator_loss=2.414, discriminator_real_loss=1.273, discriminator_fake_loss=1.142, generator_loss=30.06, generator_mel_loss=19.75, generator_kl_loss=1.975, generator_dur_loss=1.505, generator_adv_loss=2.438, generator_feat_match_loss=4.391, over 52.00 samples.], tot_loss[discriminator_loss=2.329, discriminator_real_loss=1.149, discriminator_fake_loss=1.18, generator_loss=30.31, generator_mel_loss=19.79, generator_kl_loss=1.947, generator_dur_loss=1.503, generator_adv_loss=2.454, generator_feat_match_loss=4.613, over 809.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 08:46:35,860 INFO [train.py:845] (2/4) Start epoch 395
2024-02-23 08:48:49,587 INFO [train.py:471] (2/4) Epoch 395, batch 22, global_batch_idx: 14600, batch size: 56, loss[discriminator_loss=2.551, discriminator_real_loss=1.543, discriminator_fake_loss=1.008, generator_loss=30.03, generator_mel_loss=19.72, generator_kl_loss=1.974, generator_dur_loss=1.522, generator_adv_loss=2.467, generator_feat_match_loss=4.344, over 56.00 samples.], tot_loss[discriminator_loss=2.385, discriminator_real_loss=1.205, discriminator_fake_loss=1.18, generator_loss=30.13, generator_mel_loss=19.74, generator_kl_loss=1.951, generator_dur_loss=1.496, generator_adv_loss=2.481, generator_feat_match_loss=4.459, over 1725.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 08:48:49,587 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 08:48:57,668 INFO [train.py:534] (2/4) Epoch 395, validation: discriminator_loss=2.34, discriminator_real_loss=1.187, discriminator_fake_loss=1.153, generator_loss=31.09, generator_mel_loss=20.58, generator_kl_loss=2.074, generator_dur_loss=1.496, generator_adv_loss=2.426, generator_feat_match_loss=4.516, over 100.00 samples.
2024-02-23 08:48:57,669 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 08:50:15,445 INFO [train.py:845] (2/4) Start epoch 396
2024-02-23 08:53:35,165 INFO [train.py:471] (2/4) Epoch 396, batch 35, global_batch_idx: 14650, batch size: 53, loss[discriminator_loss=2.543, discriminator_real_loss=1.2, discriminator_fake_loss=1.343, generator_loss=29.11, generator_mel_loss=19.86, generator_kl_loss=1.975, generator_dur_loss=1.497, generator_adv_loss=2.459, generator_feat_match_loss=3.326, over 53.00 samples.], tot_loss[discriminator_loss=2.435, discriminator_real_loss=1.221, discriminator_fake_loss=1.214, generator_loss=29.97, generator_mel_loss=19.81, generator_kl_loss=1.93, generator_dur_loss=1.498, generator_adv_loss=2.443, generator_feat_match_loss=4.289, over 2514.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 08:53:42,482 INFO [train.py:845] (2/4) Start epoch 397
2024-02-23 08:57:14,874 INFO [train.py:845] (2/4) Start epoch 398
2024-02-23 08:58:29,099 INFO [train.py:471] (2/4) Epoch 398, batch 11, global_batch_idx: 14700, batch size: 56, loss[discriminator_loss=2.221, discriminator_real_loss=1.219, discriminator_fake_loss=1.002, generator_loss=31.08, generator_mel_loss=20.06, generator_kl_loss=1.886, generator_dur_loss=1.472, generator_adv_loss=2.582, generator_feat_match_loss=5.078, over 56.00 samples.], tot_loss[discriminator_loss=2.431, discriminator_real_loss=1.224, discriminator_fake_loss=1.207, generator_loss=30.03, generator_mel_loss=19.94, generator_kl_loss=1.927, generator_dur_loss=1.485, generator_adv_loss=2.428, generator_feat_match_loss=4.247, over 851.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:00:44,236 INFO [train.py:845] (2/4) Start epoch 399
2024-02-23 09:03:07,703 INFO [train.py:471] (2/4) Epoch 399, batch 24, global_batch_idx: 14750, batch size: 49, loss[discriminator_loss=2.4, discriminator_real_loss=1.212, discriminator_fake_loss=1.188, generator_loss=29.41, generator_mel_loss=19.22, generator_kl_loss=1.953, generator_dur_loss=1.507, generator_adv_loss=2.477, generator_feat_match_loss=4.25, over 49.00 samples.], tot_loss[discriminator_loss=2.36, discriminator_real_loss=1.18, discriminator_fake_loss=1.179, generator_loss=30.04, generator_mel_loss=19.65, generator_kl_loss=1.932, generator_dur_loss=1.5, generator_adv_loss=2.465, generator_feat_match_loss=4.496, over 1590.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:04:16,817 INFO [train.py:845] (2/4) Start epoch 400
2024-02-23 09:07:45,281 INFO [train.py:845] (2/4) Start epoch 401
2024-02-23 09:07:57,559 INFO [train.py:471] (2/4) Epoch 401, batch 0, global_batch_idx: 14800, batch size: 79, loss[discriminator_loss=2.477, discriminator_real_loss=1.096, discriminator_fake_loss=1.382, generator_loss=29.77, generator_mel_loss=19.48, generator_kl_loss=1.972, generator_dur_loss=1.506, generator_adv_loss=2.666, generator_feat_match_loss=4.148, over 79.00 samples.], tot_loss[discriminator_loss=2.477, discriminator_real_loss=1.096, discriminator_fake_loss=1.382, generator_loss=29.77, generator_mel_loss=19.48, generator_kl_loss=1.972, generator_dur_loss=1.506, generator_adv_loss=2.666, generator_feat_match_loss=4.148, over 79.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 09:07:57,559 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 09:08:06,496 INFO [train.py:534] (2/4) Epoch 401, validation: discriminator_loss=2.494, discriminator_real_loss=1.313, discriminator_fake_loss=1.181, generator_loss=30.59, generator_mel_loss=20.49, generator_kl_loss=2.015, generator_dur_loss=1.485, generator_adv_loss=2.458, generator_feat_match_loss=4.144, over 100.00 samples.
2024-02-23 09:08:06,496 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 09:11:16,529 INFO [train.py:845] (2/4) Start epoch 402
2024-02-23 09:12:42,689 INFO [train.py:471] (2/4) Epoch 402, batch 13, global_batch_idx: 14850, batch size: 49, loss[discriminator_loss=2.551, discriminator_real_loss=1.275, discriminator_fake_loss=1.275, generator_loss=30.03, generator_mel_loss=19.98, generator_kl_loss=1.958, generator_dur_loss=1.501, generator_adv_loss=2.367, generator_feat_match_loss=4.223, over 49.00 samples.], tot_loss[discriminator_loss=2.433, discriminator_real_loss=1.208, discriminator_fake_loss=1.224, generator_loss=29.79, generator_mel_loss=19.77, generator_kl_loss=1.92, generator_dur_loss=1.485, generator_adv_loss=2.388, generator_feat_match_loss=4.232, over 1064.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 09:14:44,602 INFO [train.py:845] (2/4) Start epoch 403
2024-02-23 09:17:10,104 INFO [train.py:471] (2/4) Epoch 403, batch 26, global_batch_idx: 14900, batch size: 81, loss[discriminator_loss=2.453, discriminator_real_loss=1.239, discriminator_fake_loss=1.215, generator_loss=29.92, generator_mel_loss=20.14, generator_kl_loss=1.88, generator_dur_loss=1.48, generator_adv_loss=2.297, generator_feat_match_loss=4.125, over 81.00 samples.], tot_loss[discriminator_loss=2.479, discriminator_real_loss=1.25, discriminator_fake_loss=1.229, generator_loss=29.97, generator_mel_loss=20.06, generator_kl_loss=1.943, generator_dur_loss=1.491, generator_adv_loss=2.375, generator_feat_match_loss=4.097, over 1919.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:18:08,317 INFO [train.py:845] (2/4) Start epoch 404
2024-02-23 09:21:32,324 INFO [train.py:845] (2/4) Start epoch 405
2024-02-23 09:22:00,028 INFO [train.py:471] (2/4) Epoch 405, batch 2, global_batch_idx: 14950, batch size: 73, loss[discriminator_loss=2.32, discriminator_real_loss=1.208, discriminator_fake_loss=1.112, generator_loss=29.6, generator_mel_loss=19.41, generator_kl_loss=1.92, generator_dur_loss=1.491, generator_adv_loss=2.422, generator_feat_match_loss=4.352, over 73.00 samples.], tot_loss[discriminator_loss=2.312, discriminator_real_loss=1.155, discriminator_fake_loss=1.157, generator_loss=30.2, generator_mel_loss=19.78, generator_kl_loss=1.912, generator_dur_loss=1.489, generator_adv_loss=2.457, generator_feat_match_loss=4.564, over 252.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:25:01,911 INFO [train.py:845] (2/4) Start epoch 406
2024-02-23 09:26:35,559 INFO [train.py:471] (2/4) Epoch 406, batch 15, global_batch_idx: 15000, batch size: 64, loss[discriminator_loss=2.42, discriminator_real_loss=1.26, discriminator_fake_loss=1.16, generator_loss=29.36, generator_mel_loss=19.85, generator_kl_loss=1.908, generator_dur_loss=1.495, generator_adv_loss=2.371, generator_feat_match_loss=3.74, over 64.00 samples.], tot_loss[discriminator_loss=2.504, discriminator_real_loss=1.255, discriminator_fake_loss=1.249, generator_loss=29.22, generator_mel_loss=19.78, generator_kl_loss=1.938, generator_dur_loss=1.5, generator_adv_loss=2.309, generator_feat_match_loss=3.699, over 1053.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:26:35,560 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 09:26:44,523 INFO [train.py:534] (2/4) Epoch 406, validation: discriminator_loss=2.564, discriminator_real_loss=1.197, discriminator_fake_loss=1.367, generator_loss=29.36, generator_mel_loss=20.37, generator_kl_loss=2.038, generator_dur_loss=1.485, generator_adv_loss=2.035, generator_feat_match_loss=3.432, over 100.00 samples.
2024-02-23 09:26:44,525 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 09:28:45,060 INFO [train.py:845] (2/4) Start epoch 407
2024-02-23 09:31:30,541 INFO [train.py:471] (2/4) Epoch 407, batch 28, global_batch_idx: 15050, batch size: 51, loss[discriminator_loss=2.336, discriminator_real_loss=1.129, discriminator_fake_loss=1.208, generator_loss=29.71, generator_mel_loss=19.46, generator_kl_loss=1.896, generator_dur_loss=1.511, generator_adv_loss=2.418, generator_feat_match_loss=4.422, over 51.00 samples.], tot_loss[discriminator_loss=2.39, discriminator_real_loss=1.201, discriminator_fake_loss=1.189, generator_loss=29.71, generator_mel_loss=19.62, generator_kl_loss=1.929, generator_dur_loss=1.498, generator_adv_loss=2.402, generator_feat_match_loss=4.265, over 2028.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:32:14,930 INFO [train.py:845] (2/4) Start epoch 408
2024-02-23 09:35:39,934 INFO [train.py:845] (2/4) Start epoch 409
2024-02-23 09:36:13,459 INFO [train.py:471] (2/4) Epoch 409, batch 4, global_batch_idx: 15100, batch size: 60, loss[discriminator_loss=2.312, discriminator_real_loss=1.129, discriminator_fake_loss=1.185, generator_loss=30.08, generator_mel_loss=19.42, generator_kl_loss=1.952, generator_dur_loss=1.492, generator_adv_loss=2.404, generator_feat_match_loss=4.812, over 60.00 samples.], tot_loss[discriminator_loss=2.337, discriminator_real_loss=1.181, discriminator_fake_loss=1.157, generator_loss=30.15, generator_mel_loss=19.61, generator_kl_loss=1.938, generator_dur_loss=1.505, generator_adv_loss=2.469, generator_feat_match_loss=4.628, over 301.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:39:08,515 INFO [train.py:845] (2/4) Start epoch 410
2024-02-23 09:40:55,916 INFO [train.py:471] (2/4) Epoch 410, batch 17, global_batch_idx: 15150, batch size: 126, loss[discriminator_loss=2.297, discriminator_real_loss=1.145, discriminator_fake_loss=1.152, generator_loss=30.02, generator_mel_loss=19.83, generator_kl_loss=1.843, generator_dur_loss=1.489, generator_adv_loss=2.486, generator_feat_match_loss=4.367, over 126.00 samples.], tot_loss[discriminator_loss=2.332, discriminator_real_loss=1.167, discriminator_fake_loss=1.165, generator_loss=29.95, generator_mel_loss=19.56, generator_kl_loss=1.918, generator_dur_loss=1.489, generator_adv_loss=2.451, generator_feat_match_loss=4.528, over 1381.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 09:42:32,997 INFO [train.py:845] (2/4) Start epoch 411
2024-02-23 09:45:31,401 INFO [train.py:471] (2/4) Epoch 411, batch 30, global_batch_idx: 15200, batch size: 58, loss[discriminator_loss=2.363, discriminator_real_loss=1.114, discriminator_fake_loss=1.248, generator_loss=29.04, generator_mel_loss=18.96, generator_kl_loss=1.968, generator_dur_loss=1.499, generator_adv_loss=2.355, generator_feat_match_loss=4.266, over 58.00 samples.], tot_loss[discriminator_loss=2.449, discriminator_real_loss=1.244, discriminator_fake_loss=1.204, generator_loss=30.21, generator_mel_loss=19.72, generator_kl_loss=1.93, generator_dur_loss=1.492, generator_adv_loss=2.502, generator_feat_match_loss=4.57, over 2274.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 09:45:31,403 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 09:45:39,928 INFO [train.py:534] (2/4) Epoch 411, validation: discriminator_loss=2.628, discriminator_real_loss=1.115, discriminator_fake_loss=1.513, generator_loss=29.01, generator_mel_loss=20.05, generator_kl_loss=2.042, generator_dur_loss=1.485, generator_adv_loss=1.87, generator_feat_match_loss=3.568, over 100.00 samples.
2024-02-23 09:45:39,929 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 09:46:09,429 INFO [train.py:845] (2/4) Start epoch 412
2024-02-23 09:49:38,566 INFO [train.py:845] (2/4) Start epoch 413
2024-02-23 09:50:23,268 INFO [train.py:471] (2/4) Epoch 413, batch 6, global_batch_idx: 15250, batch size: 79, loss[discriminator_loss=2.732, discriminator_real_loss=1, discriminator_fake_loss=1.732, generator_loss=29.77, generator_mel_loss=19.85, generator_kl_loss=1.872, generator_dur_loss=1.501, generator_adv_loss=2.473, generator_feat_match_loss=4.082, over 79.00 samples.], tot_loss[discriminator_loss=2.365, discriminator_real_loss=1.192, discriminator_fake_loss=1.172, generator_loss=30.66, generator_mel_loss=19.8, generator_kl_loss=1.967, generator_dur_loss=1.488, generator_adv_loss=2.66, generator_feat_match_loss=4.746, over 571.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 09:53:07,524 INFO [train.py:845] (2/4) Start epoch 414
2024-02-23 09:54:56,394 INFO [train.py:471] (2/4) Epoch 414, batch 19, global_batch_idx: 15300, batch size: 60, loss[discriminator_loss=2.49, discriminator_real_loss=1.35, discriminator_fake_loss=1.141, generator_loss=30.14, generator_mel_loss=19.53, generator_kl_loss=1.915, generator_dur_loss=1.49, generator_adv_loss=2.561, generator_feat_match_loss=4.645, over 60.00 samples.], tot_loss[discriminator_loss=2.337, discriminator_real_loss=1.171, discriminator_fake_loss=1.166, generator_loss=30.3, generator_mel_loss=19.77, generator_kl_loss=1.922, generator_dur_loss=1.494, generator_adv_loss=2.481, generator_feat_match_loss=4.629, over 1422.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 09:56:32,821 INFO [train.py:845] (2/4) Start epoch 415
2024-02-23 09:59:37,485 INFO [train.py:471] (2/4) Epoch 415, batch 32, global_batch_idx: 15350, batch size: 53, loss[discriminator_loss=2.58, discriminator_real_loss=1.483, discriminator_fake_loss=1.097, generator_loss=30.18, generator_mel_loss=19.65, generator_kl_loss=1.893, generator_dur_loss=1.512, generator_adv_loss=2.502, generator_feat_match_loss=4.625, over 53.00 samples.], tot_loss[discriminator_loss=2.404, discriminator_real_loss=1.214, discriminator_fake_loss=1.19, generator_loss=29.75, generator_mel_loss=19.59, generator_kl_loss=1.936, generator_dur_loss=1.495, generator_adv_loss=2.423, generator_feat_match_loss=4.305, over 2298.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 10:00:03,353 INFO [train.py:845] (2/4) Start epoch 416
2024-02-23 10:03:28,418 INFO [train.py:845] (2/4) Start epoch 417
2024-02-23 10:04:23,134 INFO [train.py:471] (2/4) Epoch 417, batch 8, global_batch_idx: 15400, batch size: 69, loss[discriminator_loss=2.438, discriminator_real_loss=1.18, discriminator_fake_loss=1.258, generator_loss=29.8, generator_mel_loss=19.87, generator_kl_loss=1.977, generator_dur_loss=1.482, generator_adv_loss=2.383, generator_feat_match_loss=4.094, over 69.00 samples.], tot_loss[discriminator_loss=2.442, discriminator_real_loss=1.204, discriminator_fake_loss=1.238, generator_loss=30.04, generator_mel_loss=19.95, generator_kl_loss=1.962, generator_dur_loss=1.497, generator_adv_loss=2.375, generator_feat_match_loss=4.257, over 649.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:04:23,136 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 10:04:32,370 INFO [train.py:534] (2/4) Epoch 417, validation: discriminator_loss=2.49, discriminator_real_loss=1.219, discriminator_fake_loss=1.271, generator_loss=29.83, generator_mel_loss=20.32, generator_kl_loss=1.961, generator_dur_loss=1.487, generator_adv_loss=2.117, generator_feat_match_loss=3.94, over 100.00 samples.
2024-02-23 10:04:32,371 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 10:07:04,918 INFO [train.py:845] (2/4) Start epoch 418
2024-02-23 10:09:19,811 INFO [train.py:471] (2/4) Epoch 418, batch 21, global_batch_idx: 15450, batch size: 71, loss[discriminator_loss=2.291, discriminator_real_loss=1.146, discriminator_fake_loss=1.145, generator_loss=30.51, generator_mel_loss=19.88, generator_kl_loss=1.936, generator_dur_loss=1.496, generator_adv_loss=2.547, generator_feat_match_loss=4.656, over 71.00 samples.], tot_loss[discriminator_loss=2.383, discriminator_real_loss=1.2, discriminator_fake_loss=1.183, generator_loss=30.24, generator_mel_loss=19.83, generator_kl_loss=1.942, generator_dur_loss=1.489, generator_adv_loss=2.446, generator_feat_match_loss=4.524, over 1702.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:10:35,060 INFO [train.py:845] (2/4) Start epoch 419
2024-02-23 10:13:50,205 INFO [train.py:471] (2/4) Epoch 419, batch 34, global_batch_idx: 15500, batch size: 69, loss[discriminator_loss=2.412, discriminator_real_loss=1.158, discriminator_fake_loss=1.254, generator_loss=29.92, generator_mel_loss=19.48, generator_kl_loss=1.973, generator_dur_loss=1.48, generator_adv_loss=2.516, generator_feat_match_loss=4.469, over 69.00 samples.], tot_loss[discriminator_loss=2.373, discriminator_real_loss=1.201, discriminator_fake_loss=1.171, generator_loss=30.23, generator_mel_loss=19.75, generator_kl_loss=1.936, generator_dur_loss=1.489, generator_adv_loss=2.474, generator_feat_match_loss=4.588, over 2572.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:14:03,807 INFO [train.py:845] (2/4) Start epoch 420
2024-02-23 10:17:36,547 INFO [train.py:845] (2/4) Start epoch 421
2024-02-23 10:18:52,228 INFO [train.py:471] (2/4) Epoch 421, batch 10, global_batch_idx: 15550, batch size: 61, loss[discriminator_loss=2.332, discriminator_real_loss=1.165, discriminator_fake_loss=1.166, generator_loss=29.35, generator_mel_loss=19.06, generator_kl_loss=1.954, generator_dur_loss=1.477, generator_adv_loss=2.398, generator_feat_match_loss=4.457, over 61.00 samples.], tot_loss[discriminator_loss=2.45, discriminator_real_loss=1.283, discriminator_fake_loss=1.167, generator_loss=30, generator_mel_loss=19.6, generator_kl_loss=1.927, generator_dur_loss=1.491, generator_adv_loss=2.516, generator_feat_match_loss=4.46, over 802.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:21:09,736 INFO [train.py:845] (2/4) Start epoch 422
2024-02-23 10:23:24,994 INFO [train.py:471] (2/4) Epoch 422, batch 23, global_batch_idx: 15600, batch size: 56, loss[discriminator_loss=2.381, discriminator_real_loss=1.176, discriminator_fake_loss=1.205, generator_loss=29.97, generator_mel_loss=19.71, generator_kl_loss=1.969, generator_dur_loss=1.503, generator_adv_loss=2.498, generator_feat_match_loss=4.297, over 56.00 samples.], tot_loss[discriminator_loss=2.392, discriminator_real_loss=1.198, discriminator_fake_loss=1.194, generator_loss=30.53, generator_mel_loss=19.79, generator_kl_loss=1.946, generator_dur_loss=1.492, generator_adv_loss=2.54, generator_feat_match_loss=4.762, over 1763.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 10:23:24,995 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 10:23:33,449 INFO [train.py:534] (2/4) Epoch 422, validation: discriminator_loss=2.432, discriminator_real_loss=1.182, discriminator_fake_loss=1.25, generator_loss=30.68, generator_mel_loss=20.25, generator_kl_loss=1.997, generator_dur_loss=1.483, generator_adv_loss=2.371, generator_feat_match_loss=4.581, over 100.00 samples.
2024-02-23 10:23:33,450 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 10:24:42,687 INFO [train.py:845] (2/4) Start epoch 423
2024-02-23 10:28:14,008 INFO [train.py:471] (2/4) Epoch 423, batch 36, global_batch_idx: 15650, batch size: 63, loss[discriminator_loss=2.318, discriminator_real_loss=1.127, discriminator_fake_loss=1.191, generator_loss=30.51, generator_mel_loss=19.9, generator_kl_loss=2.003, generator_dur_loss=1.499, generator_adv_loss=2.443, generator_feat_match_loss=4.668, over 63.00 samples.], tot_loss[discriminator_loss=2.359, discriminator_real_loss=1.183, discriminator_fake_loss=1.175, generator_loss=29.98, generator_mel_loss=19.58, generator_kl_loss=1.946, generator_dur_loss=1.49, generator_adv_loss=2.457, generator_feat_match_loss=4.5, over 2699.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 10:28:14,472 INFO [train.py:845] (2/4) Start epoch 424
2024-02-23 10:31:29,225 INFO [train.py:845] (2/4) Start epoch 425
2024-02-23 10:32:42,778 INFO [train.py:471] (2/4) Epoch 425, batch 12, global_batch_idx: 15700, batch size: 71, loss[discriminator_loss=2.318, discriminator_real_loss=1.246, discriminator_fake_loss=1.072, generator_loss=30.35, generator_mel_loss=19.75, generator_kl_loss=1.984, generator_dur_loss=1.481, generator_adv_loss=2.52, generator_feat_match_loss=4.613, over 71.00 samples.], tot_loss[discriminator_loss=2.362, discriminator_real_loss=1.209, discriminator_fake_loss=1.154, generator_loss=30.04, generator_mel_loss=19.57, generator_kl_loss=1.957, generator_dur_loss=1.49, generator_adv_loss=2.487, generator_feat_match_loss=4.541, over 933.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 10:34:55,701 INFO [train.py:845] (2/4) Start epoch 426
2024-02-23 10:37:16,062 INFO [train.py:471] (2/4) Epoch 426, batch 25, global_batch_idx: 15750, batch size: 59, loss[discriminator_loss=2.52, discriminator_real_loss=1.366, discriminator_fake_loss=1.152, generator_loss=31.02, generator_mel_loss=20.09, generator_kl_loss=1.957, generator_dur_loss=1.512, generator_adv_loss=2.559, generator_feat_match_loss=4.906, over 59.00 samples.], tot_loss[discriminator_loss=2.346, discriminator_real_loss=1.186, discriminator_fake_loss=1.16, generator_loss=30.17, generator_mel_loss=19.62, generator_kl_loss=1.927, generator_dur_loss=1.492, generator_adv_loss=2.491, generator_feat_match_loss=4.641, over 1741.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 32.0
2024-02-23 10:38:24,176 INFO [train.py:845] (2/4) Start epoch 427
2024-02-23 10:41:53,447 INFO [train.py:845] (2/4) Start epoch 428
2024-02-23 10:42:13,068 INFO [train.py:471] (2/4) Epoch 428, batch 1, global_batch_idx: 15800, batch size: 126, loss[discriminator_loss=2.551, discriminator_real_loss=1.242, discriminator_fake_loss=1.309, generator_loss=29.75, generator_mel_loss=19.95, generator_kl_loss=1.903, generator_dur_loss=1.472, generator_adv_loss=2.453, generator_feat_match_loss=3.977, over 126.00 samples.], tot_loss[discriminator_loss=2.51, discriminator_real_loss=1.213, discriminator_fake_loss=1.297, generator_loss=29.64, generator_mel_loss=19.8, generator_kl_loss=1.887, generator_dur_loss=1.48, generator_adv_loss=2.435, generator_feat_match_loss=4.043, over 178.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:42:13,069 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 10:42:21,788 INFO [train.py:534] (2/4) Epoch 428, validation: discriminator_loss=2.654, discriminator_real_loss=1.347, discriminator_fake_loss=1.306, generator_loss=29.43, generator_mel_loss=20.31, generator_kl_loss=2.034, generator_dur_loss=1.486, generator_adv_loss=2.073, generator_feat_match_loss=3.529, over 100.00 samples.
2024-02-23 10:42:21,789 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 10:45:30,144 INFO [train.py:845] (2/4) Start epoch 429
2024-02-23 10:46:56,471 INFO [train.py:471] (2/4) Epoch 429, batch 14, global_batch_idx: 15850, batch size: 61, loss[discriminator_loss=2.729, discriminator_real_loss=1.674, discriminator_fake_loss=1.055, generator_loss=30.2, generator_mel_loss=19.92, generator_kl_loss=1.937, generator_dur_loss=1.508, generator_adv_loss=2.541, generator_feat_match_loss=4.285, over 61.00 samples.], tot_loss[discriminator_loss=2.448, discriminator_real_loss=1.299, discriminator_fake_loss=1.15, generator_loss=30.84, generator_mel_loss=19.86, generator_kl_loss=1.94, generator_dur_loss=1.489, generator_adv_loss=2.663, generator_feat_match_loss=4.889, over 1098.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:48:59,614 INFO [train.py:845] (2/4) Start epoch 430
2024-02-23 10:51:45,040 INFO [train.py:471] (2/4) Epoch 430, batch 27, global_batch_idx: 15900, batch size: 56, loss[discriminator_loss=2.637, discriminator_real_loss=1.429, discriminator_fake_loss=1.207, generator_loss=29.51, generator_mel_loss=19.43, generator_kl_loss=1.97, generator_dur_loss=1.481, generator_adv_loss=2.344, generator_feat_match_loss=4.281, over 56.00 samples.], tot_loss[discriminator_loss=2.41, discriminator_real_loss=1.203, discriminator_fake_loss=1.207, generator_loss=29.68, generator_mel_loss=19.66, generator_kl_loss=1.937, generator_dur_loss=1.483, generator_adv_loss=2.391, generator_feat_match_loss=4.207, over 2116.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:52:31,974 INFO [train.py:845] (2/4) Start epoch 431
2024-02-23 10:56:02,712 INFO [train.py:845] (2/4) Start epoch 432
2024-02-23 10:56:36,503 INFO [train.py:471] (2/4) Epoch 432, batch 3, global_batch_idx: 15950, batch size: 52, loss[discriminator_loss=2.443, discriminator_real_loss=1.215, discriminator_fake_loss=1.229, generator_loss=30.12, generator_mel_loss=19.81, generator_kl_loss=1.887, generator_dur_loss=1.519, generator_adv_loss=2.418, generator_feat_match_loss=4.488, over 52.00 samples.], tot_loss[discriminator_loss=2.418, discriminator_real_loss=1.2, discriminator_fake_loss=1.219, generator_loss=30.39, generator_mel_loss=20.15, generator_kl_loss=1.963, generator_dur_loss=1.482, generator_adv_loss=2.397, generator_feat_match_loss=4.394, over 389.00 samples.], cur_lr_g: 1.90e-04, cur_lr_d: 1.90e-04, grad_scale: 16.0
2024-02-23 10:59:37,820 INFO [train.py:845] (2/4) Start epoch 433
2024-02-23 11:01:10,703 INFO [train.py:471] (2/4) Epoch 433, batch 16, global_batch_idx: 16000, batch size: 56, loss[discriminator_loss=2.445, discriminator_real_loss=1.198, discriminator_fake_loss=1.247, generator_loss=30.14, generator_mel_loss=19.95, generator_kl_loss=1.951, generator_dur_loss=1.492, generator_adv_loss=2.402, generator_feat_match_loss=4.344, over 56.00 samples.], tot_loss[discriminator_loss=2.456, discriminator_real_loss=1.288, discriminator_fake_loss=1.168, generator_loss=30.44, generator_mel_loss=19.85, generator_kl_loss=1.932, generator_dur_loss=1.491, generator_adv_loss=2.583, generator_feat_match_loss=4.592, over 1014.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 32.0
2024-02-23 11:01:10,705 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 11:01:19,429 INFO [train.py:534] (2/4) Epoch 433, validation: discriminator_loss=2.396, discriminator_real_loss=1.174, discriminator_fake_loss=1.222, generator_loss=30.56, generator_mel_loss=20.32, generator_kl_loss=2.014, generator_dur_loss=1.485, generator_adv_loss=2.381, generator_feat_match_loss=4.364, over 100.00 samples.
2024-02-23 11:01:19,430 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 11:03:13,898 INFO [train.py:845] (2/4) Start epoch 434
2024-02-23 11:06:03,054 INFO [train.py:471] (2/4) Epoch 434, batch 29, global_batch_idx: 16050, batch size: 90, loss[discriminator_loss=2.66, discriminator_real_loss=1.225, discriminator_fake_loss=1.435, generator_loss=29.2, generator_mel_loss=19.94, generator_kl_loss=1.929, generator_dur_loss=1.476, generator_adv_loss=2.199, generator_feat_match_loss=3.656, over 90.00 samples.], tot_loss[discriminator_loss=2.429, discriminator_real_loss=1.211, discriminator_fake_loss=1.218, generator_loss=30.18, generator_mel_loss=20.02, generator_kl_loss=1.955, generator_dur_loss=1.488, generator_adv_loss=2.407, generator_feat_match_loss=4.314, over 2294.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:06:41,606 INFO [train.py:845] (2/4) Start epoch 435
2024-02-23 11:10:10,194 INFO [train.py:845] (2/4) Start epoch 436
2024-02-23 11:10:48,866 INFO [train.py:471] (2/4) Epoch 436, batch 5, global_batch_idx: 16100, batch size: 110, loss[discriminator_loss=2.395, discriminator_real_loss=1.188, discriminator_fake_loss=1.208, generator_loss=30.37, generator_mel_loss=19.82, generator_kl_loss=1.995, generator_dur_loss=1.483, generator_adv_loss=2.428, generator_feat_match_loss=4.641, over 110.00 samples.], tot_loss[discriminator_loss=2.452, discriminator_real_loss=1.24, discriminator_fake_loss=1.213, generator_loss=29.83, generator_mel_loss=19.64, generator_kl_loss=1.97, generator_dur_loss=1.496, generator_adv_loss=2.448, generator_feat_match_loss=4.277, over 509.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:13:39,899 INFO [train.py:845] (2/4) Start epoch 437
2024-02-23 11:15:26,173 INFO [train.py:471] (2/4) Epoch 437, batch 18, global_batch_idx: 16150, batch size: 85, loss[discriminator_loss=2.436, discriminator_real_loss=1.312, discriminator_fake_loss=1.124, generator_loss=30.47, generator_mel_loss=20.09, generator_kl_loss=1.877, generator_dur_loss=1.497, generator_adv_loss=2.402, generator_feat_match_loss=4.598, over 85.00 samples.], tot_loss[discriminator_loss=2.34, discriminator_real_loss=1.193, discriminator_fake_loss=1.147, generator_loss=30.33, generator_mel_loss=19.7, generator_kl_loss=1.93, generator_dur_loss=1.497, generator_adv_loss=2.495, generator_feat_match_loss=4.706, over 1260.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:17:01,960 INFO [train.py:845] (2/4) Start epoch 438
2024-02-23 11:20:08,838 INFO [train.py:471] (2/4) Epoch 438, batch 31, global_batch_idx: 16200, batch size: 81, loss[discriminator_loss=2.453, discriminator_real_loss=1.283, discriminator_fake_loss=1.169, generator_loss=30.46, generator_mel_loss=20.08, generator_kl_loss=1.976, generator_dur_loss=1.48, generator_adv_loss=2.486, generator_feat_match_loss=4.438, over 81.00 samples.], tot_loss[discriminator_loss=2.398, discriminator_real_loss=1.206, discriminator_fake_loss=1.193, generator_loss=30.07, generator_mel_loss=19.67, generator_kl_loss=1.934, generator_dur_loss=1.486, generator_adv_loss=2.452, generator_feat_match_loss=4.533, over 2585.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:20:08,839 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 11:20:17,500 INFO [train.py:534] (2/4) Epoch 438, validation: discriminator_loss=2.267, discriminator_real_loss=1.092, discriminator_fake_loss=1.175, generator_loss=31.61, generator_mel_loss=20.93, generator_kl_loss=2.061, generator_dur_loss=1.484, generator_adv_loss=2.504, generator_feat_match_loss=4.626, over 100.00 samples.
2024-02-23 11:20:17,500 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28234MB
2024-02-23 11:20:41,016 INFO [train.py:845] (2/4) Start epoch 439
2024-02-23 11:24:10,744 INFO [train.py:845] (2/4) Start epoch 440
2024-02-23 11:24:59,280 INFO [train.py:471] (2/4) Epoch 440, batch 7, global_batch_idx: 16250, batch size: 59, loss[discriminator_loss=2.359, discriminator_real_loss=1.206, discriminator_fake_loss=1.154, generator_loss=30.47, generator_mel_loss=19.91, generator_kl_loss=1.831, generator_dur_loss=1.488, generator_adv_loss=2.596, generator_feat_match_loss=4.645, over 59.00 samples.], tot_loss[discriminator_loss=2.465, discriminator_real_loss=1.23, discriminator_fake_loss=1.235, generator_loss=30.06, generator_mel_loss=20.03, generator_kl_loss=1.946, generator_dur_loss=1.498, generator_adv_loss=2.345, generator_feat_match_loss=4.233, over 491.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 11:27:39,591 INFO [train.py:845] (2/4) Start epoch 441
2024-02-23 11:29:42,369 INFO [train.py:471] (2/4) Epoch 441, batch 20, global_batch_idx: 16300, batch size: 67, loss[discriminator_loss=2.354, discriminator_real_loss=1.154, discriminator_fake_loss=1.199, generator_loss=30.39, generator_mel_loss=19.84, generator_kl_loss=1.961, generator_dur_loss=1.511, generator_adv_loss=2.402, generator_feat_match_loss=4.668, over 67.00 samples.], tot_loss[discriminator_loss=2.347, discriminator_real_loss=1.176, discriminator_fake_loss=1.172, generator_loss=29.94, generator_mel_loss=19.58, generator_kl_loss=1.933, generator_dur_loss=1.485, generator_adv_loss=2.438, generator_feat_match_loss=4.497, over 1718.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 11:31:02,728 INFO [train.py:845] (2/4) Start epoch 442
2024-02-23 11:34:17,102 INFO [train.py:471] (2/4) Epoch 442, batch 33, global_batch_idx: 16350, batch size: 56, loss[discriminator_loss=2.385, discriminator_real_loss=1.172, discriminator_fake_loss=1.213, generator_loss=30.35, generator_mel_loss=19.61, generator_kl_loss=1.955, generator_dur_loss=1.487, generator_adv_loss=2.438, generator_feat_match_loss=4.863, over 56.00 samples.], tot_loss[discriminator_loss=2.374, discriminator_real_loss=1.202, discriminator_fake_loss=1.172, generator_loss=30.2, generator_mel_loss=19.71, generator_kl_loss=1.926, generator_dur_loss=1.491, generator_adv_loss=2.472, generator_feat_match_loss=4.602, over 2285.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 11:34:38,721 INFO [train.py:845] (2/4) Start epoch 443
2024-02-23 11:38:07,872 INFO [train.py:845] (2/4) Start epoch 444
2024-02-23 11:39:13,896 INFO [train.py:471] (2/4) Epoch 444, batch 9, global_batch_idx: 16400, batch size: 50, loss[discriminator_loss=2.375, discriminator_real_loss=1.16, discriminator_fake_loss=1.216, generator_loss=30.44, generator_mel_loss=20.11, generator_kl_loss=1.952, generator_dur_loss=1.478, generator_adv_loss=2.373, generator_feat_match_loss=4.527, over 50.00 samples.], tot_loss[discriminator_loss=2.369, discriminator_real_loss=1.196, discriminator_fake_loss=1.173, generator_loss=30.28, generator_mel_loss=19.74, generator_kl_loss=1.931, generator_dur_loss=1.489, generator_adv_loss=2.447, generator_feat_match_loss=4.672, over 796.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:39:13,898 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 11:39:22,471 INFO [train.py:534] (2/4) Epoch 444, validation: discriminator_loss=2.351, discriminator_real_loss=1.143, discriminator_fake_loss=1.208, generator_loss=30.85, generator_mel_loss=20.45, generator_kl_loss=2.066, generator_dur_loss=1.479, generator_adv_loss=2.282, generator_feat_match_loss=4.579, over 100.00 samples.
2024-02-23 11:39:22,472 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 11:41:47,489 INFO [train.py:845] (2/4) Start epoch 445
2024-02-23 11:43:57,587 INFO [train.py:471] (2/4) Epoch 445, batch 22, global_batch_idx: 16450, batch size: 65, loss[discriminator_loss=2.314, discriminator_real_loss=1.15, discriminator_fake_loss=1.164, generator_loss=30.43, generator_mel_loss=19.81, generator_kl_loss=1.99, generator_dur_loss=1.501, generator_adv_loss=2.504, generator_feat_match_loss=4.621, over 65.00 samples.], tot_loss[discriminator_loss=2.377, discriminator_real_loss=1.189, discriminator_fake_loss=1.188, generator_loss=30.16, generator_mel_loss=19.65, generator_kl_loss=1.962, generator_dur_loss=1.485, generator_adv_loss=2.463, generator_feat_match_loss=4.601, over 1668.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:45:12,848 INFO [train.py:845] (2/4) Start epoch 446
2024-02-23 11:48:37,576 INFO [train.py:471] (2/4) Epoch 446, batch 35, global_batch_idx: 16500, batch size: 51, loss[discriminator_loss=2.367, discriminator_real_loss=1.032, discriminator_fake_loss=1.336, generator_loss=30.23, generator_mel_loss=19.55, generator_kl_loss=1.934, generator_dur_loss=1.489, generator_adv_loss=2.637, generator_feat_match_loss=4.621, over 51.00 samples.], tot_loss[discriminator_loss=2.416, discriminator_real_loss=1.222, discriminator_fake_loss=1.194, generator_loss=30.11, generator_mel_loss=19.61, generator_kl_loss=1.932, generator_dur_loss=1.49, generator_adv_loss=2.48, generator_feat_match_loss=4.598, over 2536.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:48:43,143 INFO [train.py:845] (2/4) Start epoch 447
2024-02-23 11:52:04,777 INFO [train.py:845] (2/4) Start epoch 448
2024-02-23 11:53:19,798 INFO [train.py:471] (2/4) Epoch 448, batch 11, global_batch_idx: 16550, batch size: 59, loss[discriminator_loss=2.428, discriminator_real_loss=1.23, discriminator_fake_loss=1.197, generator_loss=28.81, generator_mel_loss=19.16, generator_kl_loss=1.86, generator_dur_loss=1.488, generator_adv_loss=2.391, generator_feat_match_loss=3.91, over 59.00 samples.], tot_loss[discriminator_loss=2.456, discriminator_real_loss=1.259, discriminator_fake_loss=1.197, generator_loss=29.42, generator_mel_loss=19.43, generator_kl_loss=1.915, generator_dur_loss=1.488, generator_adv_loss=2.382, generator_feat_match_loss=4.208, over 902.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:55:35,589 INFO [train.py:845] (2/4) Start epoch 449
2024-02-23 11:58:01,134 INFO [train.py:471] (2/4) Epoch 449, batch 24, global_batch_idx: 16600, batch size: 58, loss[discriminator_loss=2.152, discriminator_real_loss=1.026, discriminator_fake_loss=1.127, generator_loss=31.28, generator_mel_loss=19.73, generator_kl_loss=1.943, generator_dur_loss=1.465, generator_adv_loss=2.596, generator_feat_match_loss=5.547, over 58.00 samples.], tot_loss[discriminator_loss=2.396, discriminator_real_loss=1.203, discriminator_fake_loss=1.192, generator_loss=29.91, generator_mel_loss=19.64, generator_kl_loss=1.952, generator_dur_loss=1.488, generator_adv_loss=2.405, generator_feat_match_loss=4.418, over 1724.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 11:58:01,135 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 11:58:09,255 INFO [train.py:534] (2/4) Epoch 449, validation: discriminator_loss=2.159, discriminator_real_loss=0.8856, discriminator_fake_loss=1.273, generator_loss=32.42, generator_mel_loss=20.98, generator_kl_loss=2.136, generator_dur_loss=1.489, generator_adv_loss=2.452, generator_feat_match_loss=5.358, over 100.00 samples.
2024-02-23 11:58:09,256 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 11:59:14,222 INFO [train.py:845] (2/4) Start epoch 450
2024-02-23 12:02:45,434 INFO [train.py:845] (2/4) Start epoch 451
2024-02-23 12:02:59,480 INFO [train.py:471] (2/4) Epoch 451, batch 0, global_batch_idx: 16650, batch size: 51, loss[discriminator_loss=2.277, discriminator_real_loss=1.097, discriminator_fake_loss=1.182, generator_loss=29.91, generator_mel_loss=19.35, generator_kl_loss=2.024, generator_dur_loss=1.491, generator_adv_loss=2.42, generator_feat_match_loss=4.633, over 51.00 samples.], tot_loss[discriminator_loss=2.277, discriminator_real_loss=1.097, discriminator_fake_loss=1.182, generator_loss=29.91, generator_mel_loss=19.35, generator_kl_loss=2.024, generator_dur_loss=1.491, generator_adv_loss=2.42, generator_feat_match_loss=4.633, over 51.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:06:14,612 INFO [train.py:845] (2/4) Start epoch 452
2024-02-23 12:07:34,502 INFO [train.py:471] (2/4) Epoch 452, batch 13, global_batch_idx: 16700, batch size: 90, loss[discriminator_loss=2.664, discriminator_real_loss=1.523, discriminator_fake_loss=1.141, generator_loss=29.91, generator_mel_loss=19.73, generator_kl_loss=1.869, generator_dur_loss=1.502, generator_adv_loss=2.277, generator_feat_match_loss=4.527, over 90.00 samples.], tot_loss[discriminator_loss=2.327, discriminator_real_loss=1.196, discriminator_fake_loss=1.131, generator_loss=30.68, generator_mel_loss=19.67, generator_kl_loss=1.916, generator_dur_loss=1.484, generator_adv_loss=2.608, generator_feat_match_loss=4.993, over 1126.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:09:36,338 INFO [train.py:845] (2/4) Start epoch 453
2024-02-23 12:12:04,320 INFO [train.py:471] (2/4) Epoch 453, batch 26, global_batch_idx: 16750, batch size: 76, loss[discriminator_loss=2.379, discriminator_real_loss=1.188, discriminator_fake_loss=1.19, generator_loss=29.84, generator_mel_loss=19.83, generator_kl_loss=1.92, generator_dur_loss=1.483, generator_adv_loss=2.369, generator_feat_match_loss=4.23, over 76.00 samples.], tot_loss[discriminator_loss=2.384, discriminator_real_loss=1.191, discriminator_fake_loss=1.194, generator_loss=29.68, generator_mel_loss=19.57, generator_kl_loss=1.923, generator_dur_loss=1.489, generator_adv_loss=2.385, generator_feat_match_loss=4.32, over 1928.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:12:59,838 INFO [train.py:845] (2/4) Start epoch 454
2024-02-23 12:16:27,037 INFO [train.py:845] (2/4) Start epoch 455
2024-02-23 12:16:48,767 INFO [train.py:471] (2/4) Epoch 455, batch 2, global_batch_idx: 16800, batch size: 61, loss[discriminator_loss=2.43, discriminator_real_loss=1.177, discriminator_fake_loss=1.252, generator_loss=29.16, generator_mel_loss=19.23, generator_kl_loss=1.963, generator_dur_loss=1.487, generator_adv_loss=2.34, generator_feat_match_loss=4.141, over 61.00 samples.], tot_loss[discriminator_loss=2.438, discriminator_real_loss=1.167, discriminator_fake_loss=1.27, generator_loss=29.33, generator_mel_loss=19.36, generator_kl_loss=1.966, generator_dur_loss=1.489, generator_adv_loss=2.347, generator_feat_match_loss=4.172, over 188.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 32.0
2024-02-23 12:16:48,767 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 12:16:58,006 INFO [train.py:534] (2/4) Epoch 455, validation: discriminator_loss=2.396, discriminator_real_loss=1.176, discriminator_fake_loss=1.221, generator_loss=30.19, generator_mel_loss=20.07, generator_kl_loss=2.061, generator_dur_loss=1.482, generator_adv_loss=2.325, generator_feat_match_loss=4.252, over 100.00 samples.
2024-02-23 12:16:58,007 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 12:20:03,326 INFO [train.py:845] (2/4) Start epoch 456
2024-02-23 12:21:42,186 INFO [train.py:471] (2/4) Epoch 456, batch 15, global_batch_idx: 16850, batch size: 71, loss[discriminator_loss=2.555, discriminator_real_loss=1.231, discriminator_fake_loss=1.323, generator_loss=30.13, generator_mel_loss=20.07, generator_kl_loss=1.975, generator_dur_loss=1.481, generator_adv_loss=2.59, generator_feat_match_loss=4.008, over 71.00 samples.], tot_loss[discriminator_loss=2.548, discriminator_real_loss=1.262, discriminator_fake_loss=1.285, generator_loss=30.01, generator_mel_loss=19.78, generator_kl_loss=1.931, generator_dur_loss=1.485, generator_adv_loss=2.475, generator_feat_match_loss=4.338, over 1221.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:23:36,290 INFO [train.py:845] (2/4) Start epoch 457
2024-02-23 12:26:08,961 INFO [train.py:471] (2/4) Epoch 457, batch 28, global_batch_idx: 16900, batch size: 85, loss[discriminator_loss=2.422, discriminator_real_loss=1.185, discriminator_fake_loss=1.236, generator_loss=29.96, generator_mel_loss=19.79, generator_kl_loss=1.955, generator_dur_loss=1.485, generator_adv_loss=2.416, generator_feat_match_loss=4.312, over 85.00 samples.], tot_loss[discriminator_loss=2.414, discriminator_real_loss=1.211, discriminator_fake_loss=1.203, generator_loss=30.03, generator_mel_loss=19.73, generator_kl_loss=1.926, generator_dur_loss=1.491, generator_adv_loss=2.459, generator_feat_match_loss=4.421, over 1920.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:26:59,596 INFO [train.py:845] (2/4) Start epoch 458
2024-02-23 12:30:23,414 INFO [train.py:845] (2/4) Start epoch 459
2024-02-23 12:30:59,991 INFO [train.py:471] (2/4) Epoch 459, batch 4, global_batch_idx: 16950, batch size: 50, loss[discriminator_loss=2.443, discriminator_real_loss=1.271, discriminator_fake_loss=1.173, generator_loss=29.93, generator_mel_loss=19.56, generator_kl_loss=1.933, generator_dur_loss=1.512, generator_adv_loss=2.473, generator_feat_match_loss=4.453, over 50.00 samples.], tot_loss[discriminator_loss=2.45, discriminator_real_loss=1.226, discriminator_fake_loss=1.224, generator_loss=29.67, generator_mel_loss=19.69, generator_kl_loss=1.946, generator_dur_loss=1.504, generator_adv_loss=2.396, generator_feat_match_loss=4.134, over 305.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:33:53,350 INFO [train.py:845] (2/4) Start epoch 460
2024-02-23 12:35:39,824 INFO [train.py:471] (2/4) Epoch 460, batch 17, global_batch_idx: 17000, batch size: 50, loss[discriminator_loss=2.543, discriminator_real_loss=1.306, discriminator_fake_loss=1.236, generator_loss=29.66, generator_mel_loss=19.68, generator_kl_loss=1.835, generator_dur_loss=1.492, generator_adv_loss=2.309, generator_feat_match_loss=4.344, over 50.00 samples.], tot_loss[discriminator_loss=2.557, discriminator_real_loss=1.293, discriminator_fake_loss=1.264, generator_loss=29.75, generator_mel_loss=19.79, generator_kl_loss=1.948, generator_dur_loss=1.492, generator_adv_loss=2.371, generator_feat_match_loss=4.145, over 1148.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:35:39,824 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 12:35:47,792 INFO [train.py:534] (2/4) Epoch 460, validation: discriminator_loss=2.368, discriminator_real_loss=1.151, discriminator_fake_loss=1.218, generator_loss=30.94, generator_mel_loss=20.58, generator_kl_loss=2.024, generator_dur_loss=1.486, generator_adv_loss=2.274, generator_feat_match_loss=4.576, over 100.00 samples.
2024-02-23 12:35:47,793 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 12:37:37,251 INFO [train.py:845] (2/4) Start epoch 461
2024-02-23 12:40:33,618 INFO [train.py:471] (2/4) Epoch 461, batch 30, global_batch_idx: 17050, batch size: 52, loss[discriminator_loss=2.34, discriminator_real_loss=1.126, discriminator_fake_loss=1.215, generator_loss=29.89, generator_mel_loss=19.5, generator_kl_loss=1.944, generator_dur_loss=1.489, generator_adv_loss=2.373, generator_feat_match_loss=4.586, over 52.00 samples.], tot_loss[discriminator_loss=2.503, discriminator_real_loss=1.238, discriminator_fake_loss=1.265, generator_loss=29.41, generator_mel_loss=19.59, generator_kl_loss=1.907, generator_dur_loss=1.491, generator_adv_loss=2.324, generator_feat_match_loss=4.1, over 2049.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:41:07,105 INFO [train.py:845] (2/4) Start epoch 462
2024-02-23 12:44:33,268 INFO [train.py:845] (2/4) Start epoch 463
2024-02-23 12:45:22,804 INFO [train.py:471] (2/4) Epoch 463, batch 6, global_batch_idx: 17100, batch size: 69, loss[discriminator_loss=2.398, discriminator_real_loss=1.276, discriminator_fake_loss=1.122, generator_loss=29.49, generator_mel_loss=19.25, generator_kl_loss=1.967, generator_dur_loss=1.485, generator_adv_loss=2.414, generator_feat_match_loss=4.375, over 69.00 samples.], tot_loss[discriminator_loss=2.451, discriminator_real_loss=1.198, discriminator_fake_loss=1.253, generator_loss=29.73, generator_mel_loss=19.72, generator_kl_loss=1.929, generator_dur_loss=1.489, generator_adv_loss=2.352, generator_feat_match_loss=4.248, over 583.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 12:48:02,698 INFO [train.py:845] (2/4) Start epoch 464
2024-02-23 12:49:55,168 INFO [train.py:471] (2/4) Epoch 464, batch 19, global_batch_idx: 17150, batch size: 59, loss[discriminator_loss=2.355, discriminator_real_loss=1.225, discriminator_fake_loss=1.131, generator_loss=30.26, generator_mel_loss=20.01, generator_kl_loss=1.927, generator_dur_loss=1.483, generator_adv_loss=2.375, generator_feat_match_loss=4.465, over 59.00 samples.], tot_loss[discriminator_loss=2.477, discriminator_real_loss=1.254, discriminator_fake_loss=1.223, generator_loss=29.65, generator_mel_loss=19.74, generator_kl_loss=1.947, generator_dur_loss=1.482, generator_adv_loss=2.324, generator_feat_match_loss=4.157, over 1370.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 12:51:27,768 INFO [train.py:845] (2/4) Start epoch 465
2024-02-23 12:54:34,602 INFO [train.py:471] (2/4) Epoch 465, batch 32, global_batch_idx: 17200, batch size: 55, loss[discriminator_loss=2.41, discriminator_real_loss=1.183, discriminator_fake_loss=1.229, generator_loss=29.56, generator_mel_loss=19.58, generator_kl_loss=1.912, generator_dur_loss=1.501, generator_adv_loss=2.283, generator_feat_match_loss=4.285, over 55.00 samples.], tot_loss[discriminator_loss=2.558, discriminator_real_loss=1.305, discriminator_fake_loss=1.254, generator_loss=29.81, generator_mel_loss=19.97, generator_kl_loss=1.936, generator_dur_loss=1.494, generator_adv_loss=2.309, generator_feat_match_loss=4.101, over 2293.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 16.0
2024-02-23 12:54:34,604 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 12:54:43,385 INFO [train.py:534] (2/4) Epoch 465, validation: discriminator_loss=2.732, discriminator_real_loss=1.186, discriminator_fake_loss=1.546, generator_loss=28.91, generator_mel_loss=20.55, generator_kl_loss=2.018, generator_dur_loss=1.491, generator_adv_loss=1.628, generator_feat_match_loss=3.224, over 100.00 samples.
2024-02-23 12:54:43,386 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 12:55:05,304 INFO [train.py:845] (2/4) Start epoch 466
2024-02-23 12:58:32,029 INFO [train.py:845] (2/4) Start epoch 467
2024-02-23 12:59:26,068 INFO [train.py:471] (2/4) Epoch 467, batch 8, global_batch_idx: 17250, batch size: 101, loss[discriminator_loss=2.5, discriminator_real_loss=1.319, discriminator_fake_loss=1.182, generator_loss=29.54, generator_mel_loss=20.07, generator_kl_loss=1.885, generator_dur_loss=1.478, generator_adv_loss=2.523, generator_feat_match_loss=3.584, over 101.00 samples.], tot_loss[discriminator_loss=2.498, discriminator_real_loss=1.225, discriminator_fake_loss=1.272, generator_loss=30.21, generator_mel_loss=20.14, generator_kl_loss=1.952, generator_dur_loss=1.486, generator_adv_loss=2.426, generator_feat_match_loss=4.207, over 661.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 13:01:58,325 INFO [train.py:845] (2/4) Start epoch 468
2024-02-23 13:04:11,064 INFO [train.py:471] (2/4) Epoch 468, batch 21, global_batch_idx: 17300, batch size: 51, loss[discriminator_loss=2.277, discriminator_real_loss=1.223, discriminator_fake_loss=1.056, generator_loss=30.84, generator_mel_loss=20.08, generator_kl_loss=1.959, generator_dur_loss=1.485, generator_adv_loss=2.568, generator_feat_match_loss=4.742, over 51.00 samples.], tot_loss[discriminator_loss=2.486, discriminator_real_loss=1.247, discriminator_fake_loss=1.239, generator_loss=29.93, generator_mel_loss=19.82, generator_kl_loss=1.919, generator_dur_loss=1.487, generator_adv_loss=2.385, generator_feat_match_loss=4.321, over 1709.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 13:05:31,885 INFO [train.py:845] (2/4) Start epoch 469
2024-02-23 13:08:54,683 INFO [train.py:471] (2/4) Epoch 469, batch 34, global_batch_idx: 17350, batch size: 73, loss[discriminator_loss=2.336, discriminator_real_loss=1.339, discriminator_fake_loss=0.9971, generator_loss=30.3, generator_mel_loss=19.38, generator_kl_loss=1.873, generator_dur_loss=1.486, generator_adv_loss=2.598, generator_feat_match_loss=4.969, over 73.00 samples.], tot_loss[discriminator_loss=2.413, discriminator_real_loss=1.216, discriminator_fake_loss=1.197, generator_loss=30.04, generator_mel_loss=19.72, generator_kl_loss=1.915, generator_dur_loss=1.489, generator_adv_loss=2.44, generator_feat_match_loss=4.473, over 2563.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 13:09:03,298 INFO [train.py:845] (2/4) Start epoch 470
2024-02-23 13:12:32,254 INFO [train.py:845] (2/4) Start epoch 471
2024-02-23 13:13:46,804 INFO [train.py:471] (2/4) Epoch 471, batch 10, global_batch_idx: 17400, batch size: 61, loss[discriminator_loss=2.359, discriminator_real_loss=1.217, discriminator_fake_loss=1.144, generator_loss=30.48, generator_mel_loss=20.12, generator_kl_loss=1.981, generator_dur_loss=1.493, generator_adv_loss=2.406, generator_feat_match_loss=4.473, over 61.00 samples.], tot_loss[discriminator_loss=2.407, discriminator_real_loss=1.187, discriminator_fake_loss=1.22, generator_loss=29.91, generator_mel_loss=19.68, generator_kl_loss=1.971, generator_dur_loss=1.485, generator_adv_loss=2.395, generator_feat_match_loss=4.376, over 819.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 13:13:46,805 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 13:13:55,060 INFO [train.py:534] (2/4) Epoch 471, validation: discriminator_loss=2.327, discriminator_real_loss=1.113, discriminator_fake_loss=1.214, generator_loss=30.3, generator_mel_loss=20.08, generator_kl_loss=1.996, generator_dur_loss=1.484, generator_adv_loss=2.293, generator_feat_match_loss=4.451, over 100.00 samples.
2024-02-23 13:13:55,060 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 13:16:14,557 INFO [train.py:845] (2/4) Start epoch 472
2024-02-23 13:18:27,717 INFO [train.py:471] (2/4) Epoch 472, batch 23, global_batch_idx: 17450, batch size: 95, loss[discriminator_loss=2.363, discriminator_real_loss=1.114, discriminator_fake_loss=1.248, generator_loss=30.33, generator_mel_loss=19.9, generator_kl_loss=1.959, generator_dur_loss=1.474, generator_adv_loss=2.344, generator_feat_match_loss=4.656, over 95.00 samples.], tot_loss[discriminator_loss=2.461, discriminator_real_loss=1.247, discriminator_fake_loss=1.214, generator_loss=30, generator_mel_loss=19.79, generator_kl_loss=1.947, generator_dur_loss=1.484, generator_adv_loss=2.394, generator_feat_match_loss=4.386, over 1697.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 13:19:48,203 INFO [train.py:845] (2/4) Start epoch 473
2024-02-23 13:23:16,888 INFO [train.py:471] (2/4) Epoch 473, batch 36, global_batch_idx: 17500, batch size: 60, loss[discriminator_loss=2.387, discriminator_real_loss=1.221, discriminator_fake_loss=1.166, generator_loss=30.21, generator_mel_loss=20.17, generator_kl_loss=2.011, generator_dur_loss=1.513, generator_adv_loss=2.326, generator_feat_match_loss=4.191, over 60.00 samples.], tot_loss[discriminator_loss=2.484, discriminator_real_loss=1.244, discriminator_fake_loss=1.239, generator_loss=29.86, generator_mel_loss=19.76, generator_kl_loss=1.93, generator_dur_loss=1.489, generator_adv_loss=2.37, generator_feat_match_loss=4.308, over 2616.00 samples.], cur_lr_g: 1.89e-04, cur_lr_d: 1.89e-04, grad_scale: 8.0
2024-02-23 13:23:17,316 INFO [train.py:845] (2/4) Start epoch 474
2024-02-23 13:26:39,988 INFO [train.py:845] (2/4) Start epoch 475
2024-02-23 13:28:02,469 INFO [train.py:471] (2/4) Epoch 475, batch 12, global_batch_idx: 17550, batch size: 64, loss[discriminator_loss=2.461, discriminator_real_loss=1.281, discriminator_fake_loss=1.18, generator_loss=30.15, generator_mel_loss=19.69, generator_kl_loss=1.941, generator_dur_loss=1.486, generator_adv_loss=2.504, generator_feat_match_loss=4.527, over 64.00 samples.], tot_loss[discriminator_loss=2.428, discriminator_real_loss=1.202, discriminator_fake_loss=1.226, generator_loss=29.95, generator_mel_loss=19.77, generator_kl_loss=1.898, generator_dur_loss=1.489, generator_adv_loss=2.362, generator_feat_match_loss=4.429, over 1004.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 13:30:09,656 INFO [train.py:845] (2/4) Start epoch 476
2024-02-23 13:32:46,134 INFO [train.py:471] (2/4) Epoch 476, batch 25, global_batch_idx: 17600, batch size: 52, loss[discriminator_loss=2.227, discriminator_real_loss=1.12, discriminator_fake_loss=1.107, generator_loss=30.72, generator_mel_loss=19.68, generator_kl_loss=1.839, generator_dur_loss=1.499, generator_adv_loss=2.535, generator_feat_match_loss=5.164, over 52.00 samples.], tot_loss[discriminator_loss=2.414, discriminator_real_loss=1.208, discriminator_fake_loss=1.206, generator_loss=30.26, generator_mel_loss=19.86, generator_kl_loss=1.92, generator_dur_loss=1.485, generator_adv_loss=2.433, generator_feat_match_loss=4.559, over 1867.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 13:32:46,135 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 13:32:54,157 INFO [train.py:534] (2/4) Epoch 476, validation: discriminator_loss=2.252, discriminator_real_loss=1.093, discriminator_fake_loss=1.159, generator_loss=31.39, generator_mel_loss=20.54, generator_kl_loss=2, generator_dur_loss=1.48, generator_adv_loss=2.423, generator_feat_match_loss=4.952, over 100.00 samples.
2024-02-23 13:32:54,158 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 13:33:47,117 INFO [train.py:845] (2/4) Start epoch 477
2024-02-23 13:37:18,440 INFO [train.py:845] (2/4) Start epoch 478
2024-02-23 13:37:38,787 INFO [train.py:471] (2/4) Epoch 478, batch 1, global_batch_idx: 17650, batch size: 90, loss[discriminator_loss=2.344, discriminator_real_loss=1.228, discriminator_fake_loss=1.116, generator_loss=30.28, generator_mel_loss=19.58, generator_kl_loss=1.921, generator_dur_loss=1.465, generator_adv_loss=2.514, generator_feat_match_loss=4.797, over 90.00 samples.], tot_loss[discriminator_loss=2.37, discriminator_real_loss=1.243, discriminator_fake_loss=1.127, generator_loss=30.02, generator_mel_loss=19.43, generator_kl_loss=1.928, generator_dur_loss=1.477, generator_adv_loss=2.521, generator_feat_match_loss=4.67, over 140.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 13:40:52,061 INFO [train.py:845] (2/4) Start epoch 479
2024-02-23 13:42:22,400 INFO [train.py:471] (2/4) Epoch 479, batch 14, global_batch_idx: 17700, batch size: 71, loss[discriminator_loss=2.605, discriminator_real_loss=1.421, discriminator_fake_loss=1.184, generator_loss=30.71, generator_mel_loss=20.01, generator_kl_loss=2.003, generator_dur_loss=1.486, generator_adv_loss=2.588, generator_feat_match_loss=4.621, over 71.00 samples.], tot_loss[discriminator_loss=2.373, discriminator_real_loss=1.209, discriminator_fake_loss=1.164, generator_loss=30.62, generator_mel_loss=19.79, generator_kl_loss=1.94, generator_dur_loss=1.486, generator_adv_loss=2.584, generator_feat_match_loss=4.821, over 1123.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 13:44:22,211 INFO [train.py:845] (2/4) Start epoch 480
2024-02-23 13:46:59,047 INFO [train.py:471] (2/4) Epoch 480, batch 27, global_batch_idx: 17750, batch size: 67, loss[discriminator_loss=2.328, discriminator_real_loss=1.193, discriminator_fake_loss=1.135, generator_loss=29.13, generator_mel_loss=18.68, generator_kl_loss=1.891, generator_dur_loss=1.509, generator_adv_loss=2.449, generator_feat_match_loss=4.605, over 67.00 samples.], tot_loss[discriminator_loss=2.338, discriminator_real_loss=1.17, discriminator_fake_loss=1.168, generator_loss=30.3, generator_mel_loss=19.53, generator_kl_loss=1.952, generator_dur_loss=1.492, generator_adv_loss=2.494, generator_feat_match_loss=4.83, over 1896.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 13:47:47,828 INFO [train.py:845] (2/4) Start epoch 481
2024-02-23 13:51:14,672 INFO [train.py:845] (2/4) Start epoch 482
2024-02-23 13:51:42,143 INFO [train.py:471] (2/4) Epoch 482, batch 3, global_batch_idx: 17800, batch size: 73, loss[discriminator_loss=2.434, discriminator_real_loss=1.207, discriminator_fake_loss=1.226, generator_loss=30.11, generator_mel_loss=20.03, generator_kl_loss=1.899, generator_dur_loss=1.495, generator_adv_loss=2.607, generator_feat_match_loss=4.082, over 73.00 samples.], tot_loss[discriminator_loss=2.509, discriminator_real_loss=1.21, discriminator_fake_loss=1.299, generator_loss=29.99, generator_mel_loss=19.87, generator_kl_loss=1.867, generator_dur_loss=1.485, generator_adv_loss=2.467, generator_feat_match_loss=4.303, over 297.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 13:51:42,144 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 13:51:50,591 INFO [train.py:534] (2/4) Epoch 482, validation: discriminator_loss=2.612, discriminator_real_loss=1.364, discriminator_fake_loss=1.248, generator_loss=29.95, generator_mel_loss=20.63, generator_kl_loss=2.013, generator_dur_loss=1.479, generator_adv_loss=2.3, generator_feat_match_loss=3.527, over 100.00 samples.
2024-02-23 13:51:50,592 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 13:54:52,916 INFO [train.py:845] (2/4) Start epoch 483
2024-02-23 13:56:33,903 INFO [train.py:471] (2/4) Epoch 483, batch 16, global_batch_idx: 17850, batch size: 110, loss[discriminator_loss=2.309, discriminator_real_loss=1.229, discriminator_fake_loss=1.08, generator_loss=31.38, generator_mel_loss=20.46, generator_kl_loss=1.962, generator_dur_loss=1.452, generator_adv_loss=2.398, generator_feat_match_loss=5.109, over 110.00 samples.], tot_loss[discriminator_loss=2.511, discriminator_real_loss=1.277, discriminator_fake_loss=1.233, generator_loss=29.95, generator_mel_loss=20.08, generator_kl_loss=1.947, generator_dur_loss=1.487, generator_adv_loss=2.316, generator_feat_match_loss=4.116, over 1154.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 13:58:24,894 INFO [train.py:845] (2/4) Start epoch 484
2024-02-23 14:01:02,909 INFO [train.py:471] (2/4) Epoch 484, batch 29, global_batch_idx: 17900, batch size: 64, loss[discriminator_loss=2.443, discriminator_real_loss=1.303, discriminator_fake_loss=1.141, generator_loss=30.02, generator_mel_loss=20, generator_kl_loss=1.865, generator_dur_loss=1.5, generator_adv_loss=2.27, generator_feat_match_loss=4.383, over 64.00 samples.], tot_loss[discriminator_loss=2.494, discriminator_real_loss=1.255, discriminator_fake_loss=1.238, generator_loss=29.77, generator_mel_loss=19.75, generator_kl_loss=1.944, generator_dur_loss=1.492, generator_adv_loss=2.356, generator_feat_match_loss=4.233, over 2084.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 14:01:47,103 INFO [train.py:845] (2/4) Start epoch 485
2024-02-23 14:05:22,192 INFO [train.py:845] (2/4) Start epoch 486
2024-02-23 14:05:58,005 INFO [train.py:471] (2/4) Epoch 486, batch 5, global_batch_idx: 17950, batch size: 81, loss[discriminator_loss=2.561, discriminator_real_loss=1.526, discriminator_fake_loss=1.034, generator_loss=29.93, generator_mel_loss=19.82, generator_kl_loss=1.837, generator_dur_loss=1.466, generator_adv_loss=2.475, generator_feat_match_loss=4.336, over 81.00 samples.], tot_loss[discriminator_loss=2.551, discriminator_real_loss=1.279, discriminator_fake_loss=1.272, generator_loss=29.56, generator_mel_loss=19.46, generator_kl_loss=1.874, generator_dur_loss=1.494, generator_adv_loss=2.405, generator_feat_match_loss=4.328, over 398.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 14:08:52,161 INFO [train.py:845] (2/4) Start epoch 487
2024-02-23 14:10:38,884 INFO [train.py:471] (2/4) Epoch 487, batch 18, global_batch_idx: 18000, batch size: 53, loss[discriminator_loss=3.559, discriminator_real_loss=2.273, discriminator_fake_loss=1.284, generator_loss=27.97, generator_mel_loss=19.47, generator_kl_loss=1.894, generator_dur_loss=1.525, generator_adv_loss=2.217, generator_feat_match_loss=2.859, over 53.00 samples.], tot_loss[discriminator_loss=2.697, discriminator_real_loss=1.383, discriminator_fake_loss=1.315, generator_loss=30.19, generator_mel_loss=20.05, generator_kl_loss=1.935, generator_dur_loss=1.486, generator_adv_loss=2.446, generator_feat_match_loss=4.268, over 1335.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 14:10:38,884 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 14:10:46,993 INFO [train.py:534] (2/4) Epoch 487, validation: discriminator_loss=3.241, discriminator_real_loss=1.899, discriminator_fake_loss=1.342, generator_loss=29.05, generator_mel_loss=20.51, generator_kl_loss=2.022, generator_dur_loss=1.481, generator_adv_loss=2.05, generator_feat_match_loss=2.987, over 100.00 samples.
2024-02-23 14:10:46,994 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 14:12:23,190 INFO [train.py:845] (2/4) Start epoch 488
2024-02-23 14:15:30,079 INFO [train.py:471] (2/4) Epoch 488, batch 31, global_batch_idx: 18050, batch size: 153, loss[discriminator_loss=2.82, discriminator_real_loss=1.372, discriminator_fake_loss=1.447, generator_loss=28.52, generator_mel_loss=19.72, generator_kl_loss=2.059, generator_dur_loss=1.446, generator_adv_loss=1.936, generator_feat_match_loss=3.355, over 153.00 samples.], tot_loss[discriminator_loss=2.833, discriminator_real_loss=1.478, discriminator_fake_loss=1.355, generator_loss=27.86, generator_mel_loss=19.6, generator_kl_loss=1.933, generator_dur_loss=1.483, generator_adv_loss=1.864, generator_feat_match_loss=2.981, over 2476.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 14:15:57,423 INFO [train.py:845] (2/4) Start epoch 489
2024-02-23 14:19:24,041 INFO [train.py:845] (2/4) Start epoch 490
2024-02-23 14:20:20,937 INFO [train.py:471] (2/4) Epoch 490, batch 7, global_batch_idx: 18100, batch size: 69, loss[discriminator_loss=3.02, discriminator_real_loss=1.485, discriminator_fake_loss=1.535, generator_loss=27.09, generator_mel_loss=19.31, generator_kl_loss=1.953, generator_dur_loss=1.472, generator_adv_loss=1.743, generator_feat_match_loss=2.609, over 69.00 samples.], tot_loss[discriminator_loss=2.739, discriminator_real_loss=1.365, discriminator_fake_loss=1.374, generator_loss=28.6, generator_mel_loss=19.88, generator_kl_loss=1.936, generator_dur_loss=1.493, generator_adv_loss=1.971, generator_feat_match_loss=3.322, over 691.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 14:22:51,565 INFO [train.py:845] (2/4) Start epoch 491
2024-02-23 14:24:42,888 INFO [train.py:471] (2/4) Epoch 491, batch 20, global_batch_idx: 18150, batch size: 65, loss[discriminator_loss=2.742, discriminator_real_loss=1.517, discriminator_fake_loss=1.226, generator_loss=28.26, generator_mel_loss=19.66, generator_kl_loss=1.897, generator_dur_loss=1.485, generator_adv_loss=1.977, generator_feat_match_loss=3.238, over 65.00 samples.], tot_loss[discriminator_loss=2.734, discriminator_real_loss=1.384, discriminator_fake_loss=1.35, generator_loss=28.28, generator_mel_loss=19.74, generator_kl_loss=1.914, generator_dur_loss=1.486, generator_adv_loss=1.937, generator_feat_match_loss=3.203, over 1389.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 14:26:20,049 INFO [train.py:845] (2/4) Start epoch 492
2024-02-23 14:29:32,725 INFO [train.py:471] (2/4) Epoch 492, batch 33, global_batch_idx: 18200, batch size: 110, loss[discriminator_loss=3.035, discriminator_real_loss=1.104, discriminator_fake_loss=1.93, generator_loss=28.18, generator_mel_loss=19.94, generator_kl_loss=1.89, generator_dur_loss=1.502, generator_adv_loss=1.706, generator_feat_match_loss=3.145, over 110.00 samples.], tot_loss[discriminator_loss=2.766, discriminator_real_loss=1.387, discriminator_fake_loss=1.379, generator_loss=28.61, generator_mel_loss=19.91, generator_kl_loss=1.936, generator_dur_loss=1.487, generator_adv_loss=1.972, generator_feat_match_loss=3.307, over 2396.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 14:29:32,726 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 14:29:41,576 INFO [train.py:534] (2/4) Epoch 492, validation: discriminator_loss=2.879, discriminator_real_loss=1.336, discriminator_fake_loss=1.543, generator_loss=29.03, generator_mel_loss=20.52, generator_kl_loss=2.06, generator_dur_loss=1.482, generator_adv_loss=1.687, generator_feat_match_loss=3.281, over 100.00 samples.
2024-02-23 14:29:41,576 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 14:29:59,428 INFO [train.py:845] (2/4) Start epoch 493
2024-02-23 14:33:30,326 INFO [train.py:845] (2/4) Start epoch 494
2024-02-23 14:34:28,224 INFO [train.py:471] (2/4) Epoch 494, batch 9, global_batch_idx: 18250, batch size: 59, loss[discriminator_loss=2.621, discriminator_real_loss=1.317, discriminator_fake_loss=1.304, generator_loss=29.17, generator_mel_loss=20.09, generator_kl_loss=1.92, generator_dur_loss=1.498, generator_adv_loss=2.289, generator_feat_match_loss=3.381, over 59.00 samples.], tot_loss[discriminator_loss=2.648, discriminator_real_loss=1.362, discriminator_fake_loss=1.285, generator_loss=29.52, generator_mel_loss=20.05, generator_kl_loss=1.924, generator_dur_loss=1.497, generator_adv_loss=2.272, generator_feat_match_loss=3.782, over 671.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 14:36:56,903 INFO [train.py:845] (2/4) Start epoch 495
2024-02-23 14:39:00,945 INFO [train.py:471] (2/4) Epoch 495, batch 22, global_batch_idx: 18300, batch size: 110, loss[discriminator_loss=2.771, discriminator_real_loss=1.203, discriminator_fake_loss=1.568, generator_loss=28.76, generator_mel_loss=20.19, generator_kl_loss=1.874, generator_dur_loss=1.465, generator_adv_loss=2.002, generator_feat_match_loss=3.227, over 110.00 samples.], tot_loss[discriminator_loss=2.687, discriminator_real_loss=1.346, discriminator_fake_loss=1.341, generator_loss=29.33, generator_mel_loss=20.23, generator_kl_loss=1.918, generator_dur_loss=1.487, generator_adv_loss=2.124, generator_feat_match_loss=3.577, over 1535.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 14:40:23,250 INFO [train.py:845] (2/4) Start epoch 496
2024-02-23 14:43:48,012 INFO [train.py:471] (2/4) Epoch 496, batch 35, global_batch_idx: 18350, batch size: 76, loss[discriminator_loss=2.707, discriminator_real_loss=1.445, discriminator_fake_loss=1.261, generator_loss=29.7, generator_mel_loss=20.19, generator_kl_loss=1.859, generator_dur_loss=1.491, generator_adv_loss=2.26, generator_feat_match_loss=3.895, over 76.00 samples.], tot_loss[discriminator_loss=2.633, discriminator_real_loss=1.328, discriminator_fake_loss=1.305, generator_loss=29.29, generator_mel_loss=20.07, generator_kl_loss=1.901, generator_dur_loss=1.485, generator_adv_loss=2.146, generator_feat_match_loss=3.684, over 2650.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 14:43:56,103 INFO [train.py:845] (2/4) Start epoch 497
2024-02-23 14:47:26,619 INFO [train.py:845] (2/4) Start epoch 498
2024-02-23 14:48:38,122 INFO [train.py:471] (2/4) Epoch 498, batch 11, global_batch_idx: 18400, batch size: 56, loss[discriminator_loss=2.627, discriminator_real_loss=1.266, discriminator_fake_loss=1.361, generator_loss=29.74, generator_mel_loss=19.85, generator_kl_loss=1.906, generator_dur_loss=1.483, generator_adv_loss=2.404, generator_feat_match_loss=4.09, over 56.00 samples.], tot_loss[discriminator_loss=2.575, discriminator_real_loss=1.281, discriminator_fake_loss=1.294, generator_loss=29.92, generator_mel_loss=20.25, generator_kl_loss=1.917, generator_dur_loss=1.484, generator_adv_loss=2.276, generator_feat_match_loss=3.997, over 919.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 14:48:38,123 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 14:48:45,781 INFO [train.py:534] (2/4) Epoch 498, validation: discriminator_loss=2.567, discriminator_real_loss=1.163, discriminator_fake_loss=1.404, generator_loss=30.24, generator_mel_loss=21.03, generator_kl_loss=2.033, generator_dur_loss=1.483, generator_adv_loss=1.825, generator_feat_match_loss=3.877, over 100.00 samples.
2024-02-23 14:48:45,782 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 14:51:04,759 INFO [train.py:845] (2/4) Start epoch 499
2024-02-23 14:53:23,356 INFO [train.py:471] (2/4) Epoch 499, batch 24, global_batch_idx: 18450, batch size: 85, loss[discriminator_loss=2.527, discriminator_real_loss=1.386, discriminator_fake_loss=1.143, generator_loss=30.69, generator_mel_loss=20.2, generator_kl_loss=1.94, generator_dur_loss=1.493, generator_adv_loss=2.576, generator_feat_match_loss=4.48, over 85.00 samples.], tot_loss[discriminator_loss=2.589, discriminator_real_loss=1.302, discriminator_fake_loss=1.287, generator_loss=29.75, generator_mel_loss=20.27, generator_kl_loss=1.927, generator_dur_loss=1.488, generator_adv_loss=2.215, generator_feat_match_loss=3.852, over 1775.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 14:54:33,390 INFO [train.py:845] (2/4) Start epoch 500
2024-02-23 14:57:59,443 INFO [train.py:845] (2/4) Start epoch 501
2024-02-23 14:58:11,302 INFO [train.py:471] (2/4) Epoch 501, batch 0, global_batch_idx: 18500, batch size: 63, loss[discriminator_loss=2.613, discriminator_real_loss=1.363, discriminator_fake_loss=1.25, generator_loss=29.68, generator_mel_loss=20.27, generator_kl_loss=2.027, generator_dur_loss=1.494, generator_adv_loss=2.146, generator_feat_match_loss=3.744, over 63.00 samples.], tot_loss[discriminator_loss=2.613, discriminator_real_loss=1.363, discriminator_fake_loss=1.25, generator_loss=29.68, generator_mel_loss=20.27, generator_kl_loss=2.027, generator_dur_loss=1.494, generator_adv_loss=2.146, generator_feat_match_loss=3.744, over 63.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:01:25,806 INFO [train.py:845] (2/4) Start epoch 502
2024-02-23 15:02:56,354 INFO [train.py:471] (2/4) Epoch 502, batch 13, global_batch_idx: 18550, batch size: 52, loss[discriminator_loss=3.086, discriminator_real_loss=1.433, discriminator_fake_loss=1.653, generator_loss=28.38, generator_mel_loss=19.86, generator_kl_loss=1.836, generator_dur_loss=1.487, generator_adv_loss=2.025, generator_feat_match_loss=3.172, over 52.00 samples.], tot_loss[discriminator_loss=2.627, discriminator_real_loss=1.337, discriminator_fake_loss=1.29, generator_loss=30.08, generator_mel_loss=20.09, generator_kl_loss=1.894, generator_dur_loss=1.473, generator_adv_loss=2.381, generator_feat_match_loss=4.245, over 1034.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:04:57,338 INFO [train.py:845] (2/4) Start epoch 503
2024-02-23 15:07:36,947 INFO [train.py:471] (2/4) Epoch 503, batch 26, global_batch_idx: 18600, batch size: 61, loss[discriminator_loss=2.361, discriminator_real_loss=1.221, discriminator_fake_loss=1.141, generator_loss=30.29, generator_mel_loss=20.18, generator_kl_loss=1.963, generator_dur_loss=1.488, generator_adv_loss=2.301, generator_feat_match_loss=4.355, over 61.00 samples.], tot_loss[discriminator_loss=2.624, discriminator_real_loss=1.326, discriminator_fake_loss=1.299, generator_loss=29.8, generator_mel_loss=20.39, generator_kl_loss=1.934, generator_dur_loss=1.487, generator_adv_loss=2.169, generator_feat_match_loss=3.818, over 2135.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:07:36,949 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 15:07:46,048 INFO [train.py:534] (2/4) Epoch 503, validation: discriminator_loss=2.356, discriminator_real_loss=1.034, discriminator_fake_loss=1.322, generator_loss=31.69, generator_mel_loss=21.55, generator_kl_loss=1.998, generator_dur_loss=1.493, generator_adv_loss=1.977, generator_feat_match_loss=4.664, over 100.00 samples.
2024-02-23 15:07:46,049 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 15:08:34,409 INFO [train.py:845] (2/4) Start epoch 504
2024-02-23 15:12:01,419 INFO [train.py:845] (2/4) Start epoch 505
2024-02-23 15:12:28,068 INFO [train.py:471] (2/4) Epoch 505, batch 2, global_batch_idx: 18650, batch size: 95, loss[discriminator_loss=2.547, discriminator_real_loss=1.347, discriminator_fake_loss=1.199, generator_loss=29.34, generator_mel_loss=19.98, generator_kl_loss=1.938, generator_dur_loss=1.477, generator_adv_loss=2.109, generator_feat_match_loss=3.836, over 95.00 samples.], tot_loss[discriminator_loss=2.59, discriminator_real_loss=1.386, discriminator_fake_loss=1.204, generator_loss=29.26, generator_mel_loss=20.01, generator_kl_loss=1.905, generator_dur_loss=1.485, generator_adv_loss=2.152, generator_feat_match_loss=3.708, over 208.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:15:30,874 INFO [train.py:845] (2/4) Start epoch 506
2024-02-23 15:16:59,823 INFO [train.py:471] (2/4) Epoch 506, batch 15, global_batch_idx: 18700, batch size: 60, loss[discriminator_loss=2.73, discriminator_real_loss=1.342, discriminator_fake_loss=1.389, generator_loss=28.29, generator_mel_loss=19.99, generator_kl_loss=2.014, generator_dur_loss=1.486, generator_adv_loss=1.892, generator_feat_match_loss=2.914, over 60.00 samples.], tot_loss[discriminator_loss=2.612, discriminator_real_loss=1.32, discriminator_fake_loss=1.293, generator_loss=29.93, generator_mel_loss=20.11, generator_kl_loss=1.934, generator_dur_loss=1.489, generator_adv_loss=2.284, generator_feat_match_loss=4.108, over 1095.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:18:59,021 INFO [train.py:845] (2/4) Start epoch 507
2024-02-23 15:21:42,927 INFO [train.py:471] (2/4) Epoch 507, batch 28, global_batch_idx: 18750, batch size: 61, loss[discriminator_loss=2.48, discriminator_real_loss=1.412, discriminator_fake_loss=1.069, generator_loss=30.76, generator_mel_loss=20.25, generator_kl_loss=1.968, generator_dur_loss=1.495, generator_adv_loss=2.539, generator_feat_match_loss=4.512, over 61.00 samples.], tot_loss[discriminator_loss=2.595, discriminator_real_loss=1.313, discriminator_fake_loss=1.282, generator_loss=29.67, generator_mel_loss=20.26, generator_kl_loss=1.939, generator_dur_loss=1.481, generator_adv_loss=2.187, generator_feat_match_loss=3.802, over 2318.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:22:27,671 INFO [train.py:845] (2/4) Start epoch 508
2024-02-23 15:26:00,210 INFO [train.py:845] (2/4) Start epoch 509
2024-02-23 15:26:32,510 INFO [train.py:471] (2/4) Epoch 509, batch 4, global_batch_idx: 18800, batch size: 60, loss[discriminator_loss=2.543, discriminator_real_loss=1.187, discriminator_fake_loss=1.357, generator_loss=30.02, generator_mel_loss=20.57, generator_kl_loss=1.966, generator_dur_loss=1.483, generator_adv_loss=2.102, generator_feat_match_loss=3.902, over 60.00 samples.], tot_loss[discriminator_loss=2.801, discriminator_real_loss=1.451, discriminator_fake_loss=1.35, generator_loss=29.51, generator_mel_loss=20.44, generator_kl_loss=1.952, generator_dur_loss=1.483, generator_adv_loss=2.004, generator_feat_match_loss=3.631, over 398.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 16.0
2024-02-23 15:26:32,511 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 15:26:41,390 INFO [train.py:534] (2/4) Epoch 509, validation: discriminator_loss=2.552, discriminator_real_loss=1.242, discriminator_fake_loss=1.309, generator_loss=30.36, generator_mel_loss=21.21, generator_kl_loss=2.052, generator_dur_loss=1.486, generator_adv_loss=2.01, generator_feat_match_loss=3.603, over 100.00 samples.
2024-02-23 15:26:41,391 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 15:29:36,047 INFO [train.py:845] (2/4) Start epoch 510
2024-02-23 15:31:18,818 INFO [train.py:471] (2/4) Epoch 510, batch 17, global_batch_idx: 18850, batch size: 71, loss[discriminator_loss=2.621, discriminator_real_loss=1.355, discriminator_fake_loss=1.267, generator_loss=30.99, generator_mel_loss=20.34, generator_kl_loss=1.927, generator_dur_loss=1.475, generator_adv_loss=2.701, generator_feat_match_loss=4.547, over 71.00 samples.], tot_loss[discriminator_loss=2.616, discriminator_real_loss=1.317, discriminator_fake_loss=1.3, generator_loss=30.06, generator_mel_loss=20.26, generator_kl_loss=1.907, generator_dur_loss=1.485, generator_adv_loss=2.349, generator_feat_match_loss=4.059, over 1307.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:33:02,439 INFO [train.py:845] (2/4) Start epoch 511
2024-02-23 15:35:59,729 INFO [train.py:471] (2/4) Epoch 511, batch 30, global_batch_idx: 18900, batch size: 85, loss[discriminator_loss=2.688, discriminator_real_loss=1.235, discriminator_fake_loss=1.453, generator_loss=28.92, generator_mel_loss=20.26, generator_kl_loss=1.867, generator_dur_loss=1.473, generator_adv_loss=1.947, generator_feat_match_loss=3.371, over 85.00 samples.], tot_loss[discriminator_loss=2.649, discriminator_real_loss=1.328, discriminator_fake_loss=1.321, generator_loss=29.58, generator_mel_loss=20.31, generator_kl_loss=1.906, generator_dur_loss=1.481, generator_adv_loss=2.14, generator_feat_match_loss=3.741, over 2352.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:36:36,303 INFO [train.py:845] (2/4) Start epoch 512
2024-02-23 15:40:03,174 INFO [train.py:845] (2/4) Start epoch 513
2024-02-23 15:40:52,831 INFO [train.py:471] (2/4) Epoch 513, batch 6, global_batch_idx: 18950, batch size: 55, loss[discriminator_loss=2.328, discriminator_real_loss=0.9663, discriminator_fake_loss=1.362, generator_loss=29.83, generator_mel_loss=20.06, generator_kl_loss=1.924, generator_dur_loss=1.522, generator_adv_loss=2.094, generator_feat_match_loss=4.223, over 55.00 samples.], tot_loss[discriminator_loss=2.554, discriminator_real_loss=1.195, discriminator_fake_loss=1.359, generator_loss=28.85, generator_mel_loss=19.99, generator_kl_loss=1.895, generator_dur_loss=1.494, generator_adv_loss=1.888, generator_feat_match_loss=3.583, over 458.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:43:35,701 INFO [train.py:845] (2/4) Start epoch 514
2024-02-23 15:45:35,051 INFO [train.py:471] (2/4) Epoch 514, batch 19, global_batch_idx: 19000, batch size: 53, loss[discriminator_loss=2.891, discriminator_real_loss=1.666, discriminator_fake_loss=1.225, generator_loss=30.7, generator_mel_loss=20.17, generator_kl_loss=1.968, generator_dur_loss=1.485, generator_adv_loss=2.799, generator_feat_match_loss=4.277, over 53.00 samples.], tot_loss[discriminator_loss=2.6, discriminator_real_loss=1.319, discriminator_fake_loss=1.281, generator_loss=30.24, generator_mel_loss=20.53, generator_kl_loss=1.947, generator_dur_loss=1.488, generator_adv_loss=2.273, generator_feat_match_loss=4.008, over 1521.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:45:35,053 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 15:45:44,705 INFO [train.py:534] (2/4) Epoch 514, validation: discriminator_loss=2.6, discriminator_real_loss=1.433, discriminator_fake_loss=1.167, generator_loss=30.88, generator_mel_loss=21.43, generator_kl_loss=1.978, generator_dur_loss=1.481, generator_adv_loss=2.18, generator_feat_match_loss=3.812, over 100.00 samples.
2024-02-23 15:45:44,706 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 15:47:14,813 INFO [train.py:845] (2/4) Start epoch 515
2024-02-23 15:50:19,826 INFO [train.py:471] (2/4) Epoch 515, batch 32, global_batch_idx: 19050, batch size: 153, loss[discriminator_loss=2.496, discriminator_real_loss=1.318, discriminator_fake_loss=1.178, generator_loss=29.63, generator_mel_loss=20.22, generator_kl_loss=2.032, generator_dur_loss=1.486, generator_adv_loss=2.062, generator_feat_match_loss=3.834, over 153.00 samples.], tot_loss[discriminator_loss=2.634, discriminator_real_loss=1.333, discriminator_fake_loss=1.301, generator_loss=29.85, generator_mel_loss=20.31, generator_kl_loss=1.899, generator_dur_loss=1.483, generator_adv_loss=2.232, generator_feat_match_loss=3.929, over 2413.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:50:47,206 INFO [train.py:845] (2/4) Start epoch 516
2024-02-23 15:54:17,552 INFO [train.py:845] (2/4) Start epoch 517
2024-02-23 15:55:11,529 INFO [train.py:471] (2/4) Epoch 517, batch 8, global_batch_idx: 19100, batch size: 95, loss[discriminator_loss=2.258, discriminator_real_loss=1.124, discriminator_fake_loss=1.135, generator_loss=30.46, generator_mel_loss=19.94, generator_kl_loss=1.885, generator_dur_loss=1.459, generator_adv_loss=2.295, generator_feat_match_loss=4.879, over 95.00 samples.], tot_loss[discriminator_loss=2.66, discriminator_real_loss=1.399, discriminator_fake_loss=1.261, generator_loss=29.95, generator_mel_loss=20.08, generator_kl_loss=1.929, generator_dur_loss=1.481, generator_adv_loss=2.33, generator_feat_match_loss=4.135, over 597.00 samples.], cur_lr_g: 1.88e-04, cur_lr_d: 1.88e-04, grad_scale: 8.0
2024-02-23 15:57:50,221 INFO [train.py:845] (2/4) Start epoch 518
2024-02-23 15:59:53,982 INFO [train.py:471] (2/4) Epoch 518, batch 21, global_batch_idx: 19150, batch size: 54, loss[discriminator_loss=2.738, discriminator_real_loss=1.484, discriminator_fake_loss=1.254, generator_loss=29.33, generator_mel_loss=19.9, generator_kl_loss=1.848, generator_dur_loss=1.505, generator_adv_loss=2.305, generator_feat_match_loss=3.779, over 54.00 samples.], tot_loss[discriminator_loss=2.638, discriminator_real_loss=1.327, discriminator_fake_loss=1.31, generator_loss=29.82, generator_mel_loss=20.34, generator_kl_loss=1.922, generator_dur_loss=1.482, generator_adv_loss=2.196, generator_feat_match_loss=3.886, over 1729.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:01:24,344 INFO [train.py:845] (2/4) Start epoch 519
2024-02-23 16:04:39,378 INFO [train.py:471] (2/4) Epoch 519, batch 34, global_batch_idx: 19200, batch size: 76, loss[discriminator_loss=2.496, discriminator_real_loss=1.17, discriminator_fake_loss=1.327, generator_loss=30.48, generator_mel_loss=20.67, generator_kl_loss=1.85, generator_dur_loss=1.499, generator_adv_loss=2.34, generator_feat_match_loss=4.129, over 76.00 samples.], tot_loss[discriminator_loss=2.61, discriminator_real_loss=1.315, discriminator_fake_loss=1.295, generator_loss=29.92, generator_mel_loss=20.42, generator_kl_loss=1.93, generator_dur_loss=1.481, generator_adv_loss=2.176, generator_feat_match_loss=3.914, over 2696.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 16.0
2024-02-23 16:04:39,380 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 16:04:48,258 INFO [train.py:534] (2/4) Epoch 519, validation: discriminator_loss=2.767, discriminator_real_loss=1.25, discriminator_fake_loss=1.517, generator_loss=29.13, generator_mel_loss=20.57, generator_kl_loss=2.007, generator_dur_loss=1.485, generator_adv_loss=1.685, generator_feat_match_loss=3.382, over 100.00 samples.
2024-02-23 16:04:48,259 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 16:04:56,830 INFO [train.py:845] (2/4) Start epoch 520
2024-02-23 16:08:29,684 INFO [train.py:845] (2/4) Start epoch 521
2024-02-23 16:09:35,832 INFO [train.py:471] (2/4) Epoch 521, batch 10, global_batch_idx: 19250, batch size: 79, loss[discriminator_loss=2.574, discriminator_real_loss=1.18, discriminator_fake_loss=1.394, generator_loss=29.72, generator_mel_loss=20.15, generator_kl_loss=1.92, generator_dur_loss=1.496, generator_adv_loss=2.346, generator_feat_match_loss=3.811, over 79.00 samples.], tot_loss[discriminator_loss=2.609, discriminator_real_loss=1.314, discriminator_fake_loss=1.295, generator_loss=29.96, generator_mel_loss=20.23, generator_kl_loss=1.901, generator_dur_loss=1.488, generator_adv_loss=2.302, generator_feat_match_loss=4.038, over 744.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:11:55,679 INFO [train.py:845] (2/4) Start epoch 522
2024-02-23 16:14:13,431 INFO [train.py:471] (2/4) Epoch 522, batch 23, global_batch_idx: 19300, batch size: 71, loss[discriminator_loss=2.773, discriminator_real_loss=1.428, discriminator_fake_loss=1.346, generator_loss=31.46, generator_mel_loss=20.66, generator_kl_loss=1.987, generator_dur_loss=1.483, generator_adv_loss=2.711, generator_feat_match_loss=4.613, over 71.00 samples.], tot_loss[discriminator_loss=2.609, discriminator_real_loss=1.325, discriminator_fake_loss=1.285, generator_loss=29.93, generator_mel_loss=20.33, generator_kl_loss=1.931, generator_dur_loss=1.483, generator_adv_loss=2.253, generator_feat_match_loss=3.933, over 1724.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:15:18,553 INFO [train.py:845] (2/4) Start epoch 523
2024-02-23 16:18:51,469 INFO [train.py:471] (2/4) Epoch 523, batch 36, global_batch_idx: 19350, batch size: 126, loss[discriminator_loss=2.668, discriminator_real_loss=1.483, discriminator_fake_loss=1.184, generator_loss=30.74, generator_mel_loss=20.37, generator_kl_loss=1.955, generator_dur_loss=1.448, generator_adv_loss=2.598, generator_feat_match_loss=4.367, over 126.00 samples.], tot_loss[discriminator_loss=2.667, discriminator_real_loss=1.343, discriminator_fake_loss=1.324, generator_loss=29.8, generator_mel_loss=20.19, generator_kl_loss=1.922, generator_dur_loss=1.483, generator_adv_loss=2.245, generator_feat_match_loss=3.956, over 2662.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:18:51,966 INFO [train.py:845] (2/4) Start epoch 524
2024-02-23 16:22:16,146 INFO [train.py:845] (2/4) Start epoch 525
2024-02-23 16:23:34,763 INFO [train.py:471] (2/4) Epoch 525, batch 12, global_batch_idx: 19400, batch size: 53, loss[discriminator_loss=2.613, discriminator_real_loss=1.408, discriminator_fake_loss=1.205, generator_loss=29.35, generator_mel_loss=20.25, generator_kl_loss=1.923, generator_dur_loss=1.499, generator_adv_loss=2.041, generator_feat_match_loss=3.637, over 53.00 samples.], tot_loss[discriminator_loss=2.639, discriminator_real_loss=1.365, discriminator_fake_loss=1.274, generator_loss=29.6, generator_mel_loss=20.4, generator_kl_loss=1.896, generator_dur_loss=1.482, generator_adv_loss=2.066, generator_feat_match_loss=3.756, over 917.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:23:34,765 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 16:23:43,236 INFO [train.py:534] (2/4) Epoch 525, validation: discriminator_loss=2.588, discriminator_real_loss=1.301, discriminator_fake_loss=1.287, generator_loss=30.91, generator_mel_loss=21.35, generator_kl_loss=2.008, generator_dur_loss=1.478, generator_adv_loss=2.089, generator_feat_match_loss=3.989, over 100.00 samples.
2024-02-23 16:23:43,237 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 16:25:55,037 INFO [train.py:845] (2/4) Start epoch 526
2024-02-23 16:28:24,316 INFO [train.py:471] (2/4) Epoch 526, batch 25, global_batch_idx: 19450, batch size: 64, loss[discriminator_loss=3.062, discriminator_real_loss=1.691, discriminator_fake_loss=1.37, generator_loss=29.23, generator_mel_loss=20.15, generator_kl_loss=1.906, generator_dur_loss=1.471, generator_adv_loss=2.207, generator_feat_match_loss=3.492, over 64.00 samples.], tot_loss[discriminator_loss=2.58, discriminator_real_loss=1.305, discriminator_fake_loss=1.275, generator_loss=30.43, generator_mel_loss=20.28, generator_kl_loss=1.918, generator_dur_loss=1.482, generator_adv_loss=2.422, generator_feat_match_loss=4.329, over 1990.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:29:24,283 INFO [train.py:845] (2/4) Start epoch 527
2024-02-23 16:32:52,129 INFO [train.py:845] (2/4) Start epoch 528
2024-02-23 16:33:08,141 INFO [train.py:471] (2/4) Epoch 528, batch 1, global_batch_idx: 19500, batch size: 56, loss[discriminator_loss=2.586, discriminator_real_loss=1.276, discriminator_fake_loss=1.311, generator_loss=29.62, generator_mel_loss=20.41, generator_kl_loss=1.93, generator_dur_loss=1.481, generator_adv_loss=2.121, generator_feat_match_loss=3.68, over 56.00 samples.], tot_loss[discriminator_loss=2.603, discriminator_real_loss=1.306, discriminator_fake_loss=1.297, generator_loss=29.4, generator_mel_loss=20.29, generator_kl_loss=1.904, generator_dur_loss=1.476, generator_adv_loss=2.048, generator_feat_match_loss=3.68, over 108.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:36:20,563 INFO [train.py:845] (2/4) Start epoch 529
2024-02-23 16:37:55,659 INFO [train.py:471] (2/4) Epoch 529, batch 14, global_batch_idx: 19550, batch size: 65, loss[discriminator_loss=2.496, discriminator_real_loss=1.192, discriminator_fake_loss=1.304, generator_loss=29.62, generator_mel_loss=19.83, generator_kl_loss=1.945, generator_dur_loss=1.494, generator_adv_loss=2.41, generator_feat_match_loss=3.941, over 65.00 samples.], tot_loss[discriminator_loss=2.542, discriminator_real_loss=1.274, discriminator_fake_loss=1.268, generator_loss=30.3, generator_mel_loss=20.24, generator_kl_loss=1.891, generator_dur_loss=1.488, generator_adv_loss=2.379, generator_feat_match_loss=4.31, over 1009.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:39:55,190 INFO [train.py:845] (2/4) Start epoch 530
2024-02-23 16:42:34,511 INFO [train.py:471] (2/4) Epoch 530, batch 27, global_batch_idx: 19600, batch size: 76, loss[discriminator_loss=2.664, discriminator_real_loss=1.453, discriminator_fake_loss=1.21, generator_loss=29.44, generator_mel_loss=20.37, generator_kl_loss=1.881, generator_dur_loss=1.491, generator_adv_loss=2.264, generator_feat_match_loss=3.434, over 76.00 samples.], tot_loss[discriminator_loss=2.647, discriminator_real_loss=1.362, discriminator_fake_loss=1.285, generator_loss=29.74, generator_mel_loss=20.21, generator_kl_loss=1.901, generator_dur_loss=1.488, generator_adv_loss=2.206, generator_feat_match_loss=3.935, over 1905.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 16.0
2024-02-23 16:42:34,513 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 16:42:42,769 INFO [train.py:534] (2/4) Epoch 530, validation: discriminator_loss=2.578, discriminator_real_loss=1.386, discriminator_fake_loss=1.192, generator_loss=30.71, generator_mel_loss=21.15, generator_kl_loss=1.929, generator_dur_loss=1.484, generator_adv_loss=2.28, generator_feat_match_loss=3.873, over 100.00 samples.
2024-02-23 16:42:42,770 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 16:43:33,239 INFO [train.py:845] (2/4) Start epoch 531
2024-02-23 16:47:00,568 INFO [train.py:845] (2/4) Start epoch 532
2024-02-23 16:47:29,102 INFO [train.py:471] (2/4) Epoch 532, batch 3, global_batch_idx: 19650, batch size: 65, loss[discriminator_loss=2.6, discriminator_real_loss=1.145, discriminator_fake_loss=1.455, generator_loss=30.42, generator_mel_loss=20.44, generator_kl_loss=1.864, generator_dur_loss=1.51, generator_adv_loss=2.559, generator_feat_match_loss=4.051, over 65.00 samples.], tot_loss[discriminator_loss=2.555, discriminator_real_loss=1.244, discriminator_fake_loss=1.311, generator_loss=30.87, generator_mel_loss=20.71, generator_kl_loss=1.909, generator_dur_loss=1.496, generator_adv_loss=2.412, generator_feat_match_loss=4.343, over 232.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:50:26,672 INFO [train.py:845] (2/4) Start epoch 533
2024-02-23 16:52:10,158 INFO [train.py:471] (2/4) Epoch 533, batch 16, global_batch_idx: 19700, batch size: 153, loss[discriminator_loss=2.611, discriminator_real_loss=1.385, discriminator_fake_loss=1.227, generator_loss=29.85, generator_mel_loss=20.16, generator_kl_loss=1.835, generator_dur_loss=1.448, generator_adv_loss=2.193, generator_feat_match_loss=4.219, over 153.00 samples.], tot_loss[discriminator_loss=2.621, discriminator_real_loss=1.339, discriminator_fake_loss=1.282, generator_loss=29.91, generator_mel_loss=20.24, generator_kl_loss=1.893, generator_dur_loss=1.481, generator_adv_loss=2.25, generator_feat_match_loss=4.043, over 1344.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:53:58,018 INFO [train.py:845] (2/4) Start epoch 534
2024-02-23 16:56:46,352 INFO [train.py:471] (2/4) Epoch 534, batch 29, global_batch_idx: 19750, batch size: 64, loss[discriminator_loss=2.91, discriminator_real_loss=1.258, discriminator_fake_loss=1.652, generator_loss=29.07, generator_mel_loss=20.05, generator_kl_loss=1.895, generator_dur_loss=1.474, generator_adv_loss=1.997, generator_feat_match_loss=3.652, over 64.00 samples.], tot_loss[discriminator_loss=2.59, discriminator_real_loss=1.299, discriminator_fake_loss=1.291, generator_loss=30.17, generator_mel_loss=20.38, generator_kl_loss=1.925, generator_dur_loss=1.485, generator_adv_loss=2.264, generator_feat_match_loss=4.112, over 2193.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 16:57:25,014 INFO [train.py:845] (2/4) Start epoch 535
2024-02-23 17:00:57,874 INFO [train.py:845] (2/4) Start epoch 536
2024-02-23 17:01:35,597 INFO [train.py:471] (2/4) Epoch 536, batch 5, global_batch_idx: 19800, batch size: 90, loss[discriminator_loss=2.611, discriminator_real_loss=1.397, discriminator_fake_loss=1.214, generator_loss=30.77, generator_mel_loss=20.57, generator_kl_loss=1.942, generator_dur_loss=1.471, generator_adv_loss=2.412, generator_feat_match_loss=4.367, over 90.00 samples.], tot_loss[discriminator_loss=2.567, discriminator_real_loss=1.294, discriminator_fake_loss=1.273, generator_loss=30.5, generator_mel_loss=20.42, generator_kl_loss=1.944, generator_dur_loss=1.475, generator_adv_loss=2.35, generator_feat_match_loss=4.318, over 484.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:01:35,599 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 17:01:44,741 INFO [train.py:534] (2/4) Epoch 536, validation: discriminator_loss=2.498, discriminator_real_loss=1.277, discriminator_fake_loss=1.22, generator_loss=31.39, generator_mel_loss=21.14, generator_kl_loss=2.032, generator_dur_loss=1.48, generator_adv_loss=2.483, generator_feat_match_loss=4.258, over 100.00 samples.
2024-02-23 17:01:44,742 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 17:04:39,972 INFO [train.py:845] (2/4) Start epoch 537
2024-02-23 17:06:25,819 INFO [train.py:471] (2/4) Epoch 537, batch 18, global_batch_idx: 19850, batch size: 65, loss[discriminator_loss=2.568, discriminator_real_loss=1.269, discriminator_fake_loss=1.3, generator_loss=29.14, generator_mel_loss=20.13, generator_kl_loss=1.868, generator_dur_loss=1.475, generator_adv_loss=2.004, generator_feat_match_loss=3.662, over 65.00 samples.], tot_loss[discriminator_loss=2.638, discriminator_real_loss=1.32, discriminator_fake_loss=1.318, generator_loss=29.9, generator_mel_loss=20.38, generator_kl_loss=1.885, generator_dur_loss=1.485, generator_adv_loss=2.189, generator_feat_match_loss=3.959, over 1371.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:08:03,320 INFO [train.py:845] (2/4) Start epoch 538
2024-02-23 17:11:02,383 INFO [train.py:471] (2/4) Epoch 538, batch 31, global_batch_idx: 19900, batch size: 101, loss[discriminator_loss=2.984, discriminator_real_loss=1.536, discriminator_fake_loss=1.449, generator_loss=29.19, generator_mel_loss=20.37, generator_kl_loss=1.87, generator_dur_loss=1.475, generator_adv_loss=1.89, generator_feat_match_loss=3.578, over 101.00 samples.], tot_loss[discriminator_loss=2.594, discriminator_real_loss=1.315, discriminator_fake_loss=1.28, generator_loss=30.27, generator_mel_loss=20.37, generator_kl_loss=1.906, generator_dur_loss=1.481, generator_adv_loss=2.301, generator_feat_match_loss=4.21, over 2348.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:11:34,932 INFO [train.py:845] (2/4) Start epoch 539
2024-02-23 17:15:04,992 INFO [train.py:845] (2/4) Start epoch 540
2024-02-23 17:16:02,510 INFO [train.py:471] (2/4) Epoch 540, batch 7, global_batch_idx: 19950, batch size: 90, loss[discriminator_loss=2.613, discriminator_real_loss=1.34, discriminator_fake_loss=1.274, generator_loss=30.29, generator_mel_loss=20.6, generator_kl_loss=2.031, generator_dur_loss=1.494, generator_adv_loss=2.309, generator_feat_match_loss=3.859, over 90.00 samples.], tot_loss[discriminator_loss=2.563, discriminator_real_loss=1.288, discriminator_fake_loss=1.276, generator_loss=30.3, generator_mel_loss=20.57, generator_kl_loss=1.933, generator_dur_loss=1.478, generator_adv_loss=2.209, generator_feat_match_loss=4.109, over 696.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:18:31,562 INFO [train.py:845] (2/4) Start epoch 541
2024-02-23 17:20:31,847 INFO [train.py:471] (2/4) Epoch 541, batch 20, global_batch_idx: 20000, batch size: 65, loss[discriminator_loss=2.457, discriminator_real_loss=1.261, discriminator_fake_loss=1.197, generator_loss=30.53, generator_mel_loss=20.11, generator_kl_loss=1.816, generator_dur_loss=1.481, generator_adv_loss=2.375, generator_feat_match_loss=4.75, over 65.00 samples.], tot_loss[discriminator_loss=2.581, discriminator_real_loss=1.313, discriminator_fake_loss=1.268, generator_loss=30.05, generator_mel_loss=20.07, generator_kl_loss=1.916, generator_dur_loss=1.49, generator_adv_loss=2.307, generator_feat_match_loss=4.26, over 1452.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 16.0
2024-02-23 17:20:31,849 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 17:20:40,068 INFO [train.py:534] (2/4) Epoch 541, validation: discriminator_loss=2.334, discriminator_real_loss=1.144, discriminator_fake_loss=1.19, generator_loss=31.42, generator_mel_loss=21, generator_kl_loss=2.015, generator_dur_loss=1.483, generator_adv_loss=2.197, generator_feat_match_loss=4.727, over 100.00 samples.
2024-02-23 17:20:40,069 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 17:22:10,818 INFO [train.py:845] (2/4) Start epoch 542
2024-02-23 17:25:22,892 INFO [train.py:471] (2/4) Epoch 542, batch 33, global_batch_idx: 20050, batch size: 52, loss[discriminator_loss=2.559, discriminator_real_loss=1.358, discriminator_fake_loss=1.201, generator_loss=29.89, generator_mel_loss=20.19, generator_kl_loss=1.907, generator_dur_loss=1.476, generator_adv_loss=2.08, generator_feat_match_loss=4.234, over 52.00 samples.], tot_loss[discriminator_loss=2.584, discriminator_real_loss=1.316, discriminator_fake_loss=1.268, generator_loss=29.91, generator_mel_loss=20.27, generator_kl_loss=1.919, generator_dur_loss=1.482, generator_adv_loss=2.193, generator_feat_match_loss=4.049, over 2464.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:25:39,080 INFO [train.py:845] (2/4) Start epoch 543
2024-02-23 17:29:10,448 INFO [train.py:845] (2/4) Start epoch 544
2024-02-23 17:30:11,853 INFO [train.py:471] (2/4) Epoch 544, batch 9, global_batch_idx: 20100, batch size: 65, loss[discriminator_loss=2.676, discriminator_real_loss=1.434, discriminator_fake_loss=1.241, generator_loss=30.55, generator_mel_loss=20.28, generator_kl_loss=1.924, generator_dur_loss=1.503, generator_adv_loss=2.416, generator_feat_match_loss=4.426, over 65.00 samples.], tot_loss[discriminator_loss=2.685, discriminator_real_loss=1.35, discriminator_fake_loss=1.335, generator_loss=29.89, generator_mel_loss=20.26, generator_kl_loss=1.905, generator_dur_loss=1.48, generator_adv_loss=2.239, generator_feat_match_loss=4.01, over 816.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:32:43,818 INFO [train.py:845] (2/4) Start epoch 545
2024-02-23 17:34:57,798 INFO [train.py:471] (2/4) Epoch 545, batch 22, global_batch_idx: 20150, batch size: 65, loss[discriminator_loss=2.754, discriminator_real_loss=1.56, discriminator_fake_loss=1.194, generator_loss=28.73, generator_mel_loss=19.77, generator_kl_loss=1.94, generator_dur_loss=1.482, generator_adv_loss=2.047, generator_feat_match_loss=3.49, over 65.00 samples.], tot_loss[discriminator_loss=2.638, discriminator_real_loss=1.325, discriminator_fake_loss=1.312, generator_loss=29.54, generator_mel_loss=20.08, generator_kl_loss=1.892, generator_dur_loss=1.485, generator_adv_loss=2.142, generator_feat_match_loss=3.948, over 1648.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:36:17,382 INFO [train.py:845] (2/4) Start epoch 546
2024-02-23 17:39:41,165 INFO [train.py:471] (2/4) Epoch 546, batch 35, global_batch_idx: 20200, batch size: 67, loss[discriminator_loss=2.605, discriminator_real_loss=1.396, discriminator_fake_loss=1.209, generator_loss=30.21, generator_mel_loss=20, generator_kl_loss=2.044, generator_dur_loss=1.501, generator_adv_loss=2.246, generator_feat_match_loss=4.414, over 67.00 samples.], tot_loss[discriminator_loss=2.627, discriminator_real_loss=1.327, discriminator_fake_loss=1.3, generator_loss=29.78, generator_mel_loss=20.29, generator_kl_loss=1.915, generator_dur_loss=1.483, generator_adv_loss=2.151, generator_feat_match_loss=3.94, over 2557.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:39:41,167 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 17:39:49,525 INFO [train.py:534] (2/4) Epoch 546, validation: discriminator_loss=2.47, discriminator_real_loss=1.181, discriminator_fake_loss=1.289, generator_loss=30.19, generator_mel_loss=20.64, generator_kl_loss=2.002, generator_dur_loss=1.483, generator_adv_loss=2.068, generator_feat_match_loss=3.988, over 100.00 samples.
2024-02-23 17:39:49,526 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 17:39:53,914 INFO [train.py:845] (2/4) Start epoch 547
2024-02-23 17:43:27,279 INFO [train.py:845] (2/4) Start epoch 548
2024-02-23 17:44:38,660 INFO [train.py:471] (2/4) Epoch 548, batch 11, global_batch_idx: 20250, batch size: 101, loss[discriminator_loss=2.553, discriminator_real_loss=1.235, discriminator_fake_loss=1.317, generator_loss=29.99, generator_mel_loss=20.47, generator_kl_loss=1.827, generator_dur_loss=1.474, generator_adv_loss=2.148, generator_feat_match_loss=4.07, over 101.00 samples.], tot_loss[discriminator_loss=2.56, discriminator_real_loss=1.271, discriminator_fake_loss=1.289, generator_loss=30.12, generator_mel_loss=20.42, generator_kl_loss=1.905, generator_dur_loss=1.49, generator_adv_loss=2.209, generator_feat_match_loss=4.089, over 792.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:46:59,593 INFO [train.py:845] (2/4) Start epoch 549
2024-02-23 17:49:18,240 INFO [train.py:471] (2/4) Epoch 549, batch 24, global_batch_idx: 20300, batch size: 65, loss[discriminator_loss=3, discriminator_real_loss=1.597, discriminator_fake_loss=1.402, generator_loss=29.28, generator_mel_loss=20.19, generator_kl_loss=1.988, generator_dur_loss=1.47, generator_adv_loss=2.061, generator_feat_match_loss=3.578, over 65.00 samples.], tot_loss[discriminator_loss=2.602, discriminator_real_loss=1.317, discriminator_fake_loss=1.285, generator_loss=30.11, generator_mel_loss=20.15, generator_kl_loss=1.919, generator_dur_loss=1.475, generator_adv_loss=2.296, generator_feat_match_loss=4.274, over 1985.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:50:23,989 INFO [train.py:845] (2/4) Start epoch 550
2024-02-23 17:53:52,699 INFO [train.py:845] (2/4) Start epoch 551
2024-02-23 17:54:06,380 INFO [train.py:471] (2/4) Epoch 551, batch 0, global_batch_idx: 20350, batch size: 64, loss[discriminator_loss=2.621, discriminator_real_loss=1.326, discriminator_fake_loss=1.294, generator_loss=29.41, generator_mel_loss=20.26, generator_kl_loss=1.83, generator_dur_loss=1.47, generator_adv_loss=2.061, generator_feat_match_loss=3.785, over 64.00 samples.], tot_loss[discriminator_loss=2.621, discriminator_real_loss=1.326, discriminator_fake_loss=1.294, generator_loss=29.41, generator_mel_loss=20.26, generator_kl_loss=1.83, generator_dur_loss=1.47, generator_adv_loss=2.061, generator_feat_match_loss=3.785, over 64.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 17:57:26,742 INFO [train.py:845] (2/4) Start epoch 552
2024-02-23 17:58:50,420 INFO [train.py:471] (2/4) Epoch 552, batch 13, global_batch_idx: 20400, batch size: 67, loss[discriminator_loss=2.389, discriminator_real_loss=1.19, discriminator_fake_loss=1.198, generator_loss=30.16, generator_mel_loss=20.15, generator_kl_loss=1.89, generator_dur_loss=1.491, generator_adv_loss=2.139, generator_feat_match_loss=4.484, over 67.00 samples.], tot_loss[discriminator_loss=2.729, discriminator_real_loss=1.41, discriminator_fake_loss=1.319, generator_loss=29.84, generator_mel_loss=20.28, generator_kl_loss=1.92, generator_dur_loss=1.479, generator_adv_loss=2.162, generator_feat_match_loss=3.999, over 1016.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 16.0
2024-02-23 17:58:50,421 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 17:58:59,559 INFO [train.py:534] (2/4) Epoch 552, validation: discriminator_loss=2.474, discriminator_real_loss=1.045, discriminator_fake_loss=1.429, generator_loss=30.46, generator_mel_loss=20.85, generator_kl_loss=2.012, generator_dur_loss=1.487, generator_adv_loss=1.767, generator_feat_match_loss=4.343, over 100.00 samples.
2024-02-23 17:58:59,560 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 18:01:08,062 INFO [train.py:845] (2/4) Start epoch 553
2024-02-23 18:03:45,243 INFO [train.py:471] (2/4) Epoch 553, batch 26, global_batch_idx: 20450, batch size: 50, loss[discriminator_loss=2.453, discriminator_real_loss=1.187, discriminator_fake_loss=1.267, generator_loss=29.69, generator_mel_loss=19.8, generator_kl_loss=1.925, generator_dur_loss=1.488, generator_adv_loss=2.23, generator_feat_match_loss=4.242, over 50.00 samples.], tot_loss[discriminator_loss=2.589, discriminator_real_loss=1.321, discriminator_fake_loss=1.268, generator_loss=29.61, generator_mel_loss=20.2, generator_kl_loss=1.907, generator_dur_loss=1.481, generator_adv_loss=2.108, generator_feat_match_loss=3.917, over 1932.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 16.0
2024-02-23 18:04:40,842 INFO [train.py:845] (2/4) Start epoch 554
2024-02-23 18:08:06,584 INFO [train.py:845] (2/4) Start epoch 555
2024-02-23 18:08:31,440 INFO [train.py:471] (2/4) Epoch 555, batch 2, global_batch_idx: 20500, batch size: 95, loss[discriminator_loss=2.59, discriminator_real_loss=1.357, discriminator_fake_loss=1.233, generator_loss=29.83, generator_mel_loss=20.22, generator_kl_loss=1.896, generator_dur_loss=1.459, generator_adv_loss=2.293, generator_feat_match_loss=3.965, over 95.00 samples.], tot_loss[discriminator_loss=2.588, discriminator_real_loss=1.324, discriminator_fake_loss=1.264, generator_loss=29.63, generator_mel_loss=20.12, generator_kl_loss=1.922, generator_dur_loss=1.472, generator_adv_loss=2.199, generator_feat_match_loss=3.919, over 270.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 16.0
2024-02-23 18:11:32,090 INFO [train.py:845] (2/4) Start epoch 556
2024-02-23 18:13:05,851 INFO [train.py:471] (2/4) Epoch 556, batch 15, global_batch_idx: 20550, batch size: 52, loss[discriminator_loss=2.676, discriminator_real_loss=1.385, discriminator_fake_loss=1.291, generator_loss=28.85, generator_mel_loss=19.45, generator_kl_loss=1.9, generator_dur_loss=1.482, generator_adv_loss=2.154, generator_feat_match_loss=3.861, over 52.00 samples.], tot_loss[discriminator_loss=2.626, discriminator_real_loss=1.345, discriminator_fake_loss=1.281, generator_loss=30.11, generator_mel_loss=20.18, generator_kl_loss=1.891, generator_dur_loss=1.48, generator_adv_loss=2.288, generator_feat_match_loss=4.27, over 1239.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 18:15:02,135 INFO [train.py:845] (2/4) Start epoch 557
2024-02-23 18:17:43,643 INFO [train.py:471] (2/4) Epoch 557, batch 28, global_batch_idx: 20600, batch size: 49, loss[discriminator_loss=2.451, discriminator_real_loss=1.304, discriminator_fake_loss=1.147, generator_loss=30.4, generator_mel_loss=20.11, generator_kl_loss=1.863, generator_dur_loss=1.491, generator_adv_loss=2.285, generator_feat_match_loss=4.656, over 49.00 samples.], tot_loss[discriminator_loss=2.568, discriminator_real_loss=1.292, discriminator_fake_loss=1.277, generator_loss=30.28, generator_mel_loss=20.37, generator_kl_loss=1.936, generator_dur_loss=1.483, generator_adv_loss=2.259, generator_feat_match_loss=4.237, over 2091.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 18:17:43,645 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 18:17:52,536 INFO [train.py:534] (2/4) Epoch 557, validation: discriminator_loss=2.607, discriminator_real_loss=1.101, discriminator_fake_loss=1.506, generator_loss=30.57, generator_mel_loss=21.3, generator_kl_loss=2.043, generator_dur_loss=1.48, generator_adv_loss=1.69, generator_feat_match_loss=4.053, over 100.00 samples.
2024-02-23 18:17:52,536 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 18:18:35,537 INFO [train.py:845] (2/4) Start epoch 558
2024-02-23 18:22:02,082 INFO [train.py:845] (2/4) Start epoch 559
2024-02-23 18:22:40,404 INFO [train.py:471] (2/4) Epoch 559, batch 4, global_batch_idx: 20650, batch size: 59, loss[discriminator_loss=2.555, discriminator_real_loss=1.186, discriminator_fake_loss=1.369, generator_loss=30.62, generator_mel_loss=20.75, generator_kl_loss=1.92, generator_dur_loss=1.498, generator_adv_loss=2.164, generator_feat_match_loss=4.289, over 59.00 samples.], tot_loss[discriminator_loss=2.598, discriminator_real_loss=1.288, discriminator_fake_loss=1.31, generator_loss=30.07, generator_mel_loss=20.35, generator_kl_loss=1.895, generator_dur_loss=1.489, generator_adv_loss=2.204, generator_feat_match_loss=4.13, over 339.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 18:25:28,576 INFO [train.py:845] (2/4) Start epoch 560
2024-02-23 18:27:12,638 INFO [train.py:471] (2/4) Epoch 560, batch 17, global_batch_idx: 20700, batch size: 76, loss[discriminator_loss=2.531, discriminator_real_loss=1.221, discriminator_fake_loss=1.312, generator_loss=30.79, generator_mel_loss=20.83, generator_kl_loss=1.865, generator_dur_loss=1.51, generator_adv_loss=2.254, generator_feat_match_loss=4.328, over 76.00 samples.], tot_loss[discriminator_loss=2.525, discriminator_real_loss=1.266, discriminator_fake_loss=1.259, generator_loss=30.11, generator_mel_loss=20.06, generator_kl_loss=1.874, generator_dur_loss=1.482, generator_adv_loss=2.325, generator_feat_match_loss=4.37, over 1347.00 samples.], cur_lr_g: 1.87e-04, cur_lr_d: 1.87e-04, grad_scale: 8.0
2024-02-23 18:28:59,879 INFO [train.py:845] (2/4) Start epoch 561
2024-02-23 18:31:52,356 INFO [train.py:471] (2/4) Epoch 561, batch 30, global_batch_idx: 20750, batch size: 71, loss[discriminator_loss=2.494, discriminator_real_loss=1.203, discriminator_fake_loss=1.291, generator_loss=30.66, generator_mel_loss=20.25, generator_kl_loss=1.906, generator_dur_loss=1.481, generator_adv_loss=2.438, generator_feat_match_loss=4.578, over 71.00 samples.], tot_loss[discriminator_loss=2.561, discriminator_real_loss=1.284, discriminator_fake_loss=1.277, generator_loss=30.32, generator_mel_loss=20.36, generator_kl_loss=1.91, generator_dur_loss=1.476, generator_adv_loss=2.287, generator_feat_match_loss=4.282, over 2442.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 18:32:29,587 INFO [train.py:845] (2/4) Start epoch 562
2024-02-23 18:35:56,706 INFO [train.py:845] (2/4) Start epoch 563
2024-02-23 18:36:41,387 INFO [train.py:471] (2/4) Epoch 563, batch 6, global_batch_idx: 20800, batch size: 52, loss[discriminator_loss=2.57, discriminator_real_loss=1.246, discriminator_fake_loss=1.323, generator_loss=29.32, generator_mel_loss=19.9, generator_kl_loss=1.876, generator_dur_loss=1.47, generator_adv_loss=2.104, generator_feat_match_loss=3.965, over 52.00 samples.], tot_loss[discriminator_loss=2.607, discriminator_real_loss=1.359, discriminator_fake_loss=1.249, generator_loss=29.72, generator_mel_loss=20.03, generator_kl_loss=1.932, generator_dur_loss=1.49, generator_adv_loss=2.243, generator_feat_match_loss=4.027, over 416.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 16.0
2024-02-23 18:36:41,388 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 18:36:50,245 INFO [train.py:534] (2/4) Epoch 563, validation: discriminator_loss=2.591, discriminator_real_loss=1.133, discriminator_fake_loss=1.458, generator_loss=30.62, generator_mel_loss=21.25, generator_kl_loss=1.977, generator_dur_loss=1.474, generator_adv_loss=1.843, generator_feat_match_loss=4.08, over 100.00 samples.
2024-02-23 18:36:50,246 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 18:39:39,147 INFO [train.py:845] (2/4) Start epoch 564
2024-02-23 18:41:38,066 INFO [train.py:471] (2/4) Epoch 564, batch 19, global_batch_idx: 20850, batch size: 67, loss[discriminator_loss=2.652, discriminator_real_loss=1.399, discriminator_fake_loss=1.252, generator_loss=30.62, generator_mel_loss=20.2, generator_kl_loss=1.878, generator_dur_loss=1.485, generator_adv_loss=2.598, generator_feat_match_loss=4.461, over 67.00 samples.], tot_loss[discriminator_loss=2.577, discriminator_real_loss=1.303, discriminator_fake_loss=1.274, generator_loss=30.32, generator_mel_loss=20.26, generator_kl_loss=1.88, generator_dur_loss=1.478, generator_adv_loss=2.342, generator_feat_match_loss=4.364, over 1553.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 18:43:09,256 INFO [train.py:845] (2/4) Start epoch 565
2024-02-23 18:46:14,402 INFO [train.py:471] (2/4) Epoch 565, batch 32, global_batch_idx: 20900, batch size: 50, loss[discriminator_loss=2.414, discriminator_real_loss=1.232, discriminator_fake_loss=1.182, generator_loss=30.94, generator_mel_loss=20.39, generator_kl_loss=1.929, generator_dur_loss=1.482, generator_adv_loss=2.25, generator_feat_match_loss=4.895, over 50.00 samples.], tot_loss[discriminator_loss=2.576, discriminator_real_loss=1.304, discriminator_fake_loss=1.272, generator_loss=30.28, generator_mel_loss=20.45, generator_kl_loss=1.921, generator_dur_loss=1.48, generator_adv_loss=2.227, generator_feat_match_loss=4.195, over 2356.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 18:46:37,697 INFO [train.py:845] (2/4) Start epoch 566
2024-02-23 18:50:03,104 INFO [train.py:845] (2/4) Start epoch 567
2024-02-23 18:50:57,738 INFO [train.py:471] (2/4) Epoch 567, batch 8, global_batch_idx: 20950, batch size: 69, loss[discriminator_loss=2.523, discriminator_real_loss=1.277, discriminator_fake_loss=1.246, generator_loss=30.36, generator_mel_loss=20.21, generator_kl_loss=1.916, generator_dur_loss=1.467, generator_adv_loss=2.283, generator_feat_match_loss=4.488, over 69.00 samples.], tot_loss[discriminator_loss=2.542, discriminator_real_loss=1.306, discriminator_fake_loss=1.236, generator_loss=30.32, generator_mel_loss=20.44, generator_kl_loss=1.921, generator_dur_loss=1.483, generator_adv_loss=2.253, generator_feat_match_loss=4.221, over 661.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 18:53:30,694 INFO [train.py:845] (2/4) Start epoch 568
2024-02-23 18:55:34,843 INFO [train.py:471] (2/4) Epoch 568, batch 21, global_batch_idx: 21000, batch size: 52, loss[discriminator_loss=2.496, discriminator_real_loss=1.199, discriminator_fake_loss=1.298, generator_loss=30.65, generator_mel_loss=20.29, generator_kl_loss=1.819, generator_dur_loss=1.473, generator_adv_loss=2.293, generator_feat_match_loss=4.773, over 52.00 samples.], tot_loss[discriminator_loss=2.548, discriminator_real_loss=1.312, discriminator_fake_loss=1.236, generator_loss=30.25, generator_mel_loss=20.06, generator_kl_loss=1.904, generator_dur_loss=1.48, generator_adv_loss=2.364, generator_feat_match_loss=4.438, over 1486.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 18:55:34,844 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 18:55:43,201 INFO [train.py:534] (2/4) Epoch 568, validation: discriminator_loss=2.541, discriminator_real_loss=1.119, discriminator_fake_loss=1.422, generator_loss=29.93, generator_mel_loss=20.55, generator_kl_loss=1.977, generator_dur_loss=1.476, generator_adv_loss=1.76, generator_feat_match_loss=4.171, over 100.00 samples.
2024-02-23 18:55:43,203 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 18:57:07,292 INFO [train.py:845] (2/4) Start epoch 569
2024-02-23 19:00:29,518 INFO [train.py:471] (2/4) Epoch 569, batch 34, global_batch_idx: 21050, batch size: 79, loss[discriminator_loss=2.58, discriminator_real_loss=1.301, discriminator_fake_loss=1.279, generator_loss=30.18, generator_mel_loss=20.65, generator_kl_loss=1.91, generator_dur_loss=1.483, generator_adv_loss=2.104, generator_feat_match_loss=4.035, over 79.00 samples.], tot_loss[discriminator_loss=2.562, discriminator_real_loss=1.295, discriminator_fake_loss=1.267, generator_loss=29.96, generator_mel_loss=20.05, generator_kl_loss=1.921, generator_dur_loss=1.481, generator_adv_loss=2.25, generator_feat_match_loss=4.254, over 2518.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:00:39,515 INFO [train.py:845] (2/4) Start epoch 570
2024-02-23 19:04:09,178 INFO [train.py:845] (2/4) Start epoch 571
2024-02-23 19:05:24,869 INFO [train.py:471] (2/4) Epoch 571, batch 10, global_batch_idx: 21100, batch size: 59, loss[discriminator_loss=2.684, discriminator_real_loss=1.418, discriminator_fake_loss=1.265, generator_loss=29.67, generator_mel_loss=20.44, generator_kl_loss=1.937, generator_dur_loss=1.508, generator_adv_loss=2.141, generator_feat_match_loss=3.646, over 59.00 samples.], tot_loss[discriminator_loss=2.633, discriminator_real_loss=1.282, discriminator_fake_loss=1.351, generator_loss=29.8, generator_mel_loss=20.06, generator_kl_loss=1.896, generator_dur_loss=1.483, generator_adv_loss=2.192, generator_feat_match_loss=4.176, over 757.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:07:35,049 INFO [train.py:845] (2/4) Start epoch 572
2024-02-23 19:09:58,266 INFO [train.py:471] (2/4) Epoch 572, batch 23, global_batch_idx: 21150, batch size: 58, loss[discriminator_loss=2.582, discriminator_real_loss=1.195, discriminator_fake_loss=1.387, generator_loss=29.99, generator_mel_loss=20.26, generator_kl_loss=1.852, generator_dur_loss=1.485, generator_adv_loss=1.98, generator_feat_match_loss=4.414, over 58.00 samples.], tot_loss[discriminator_loss=2.628, discriminator_real_loss=1.339, discriminator_fake_loss=1.289, generator_loss=30.06, generator_mel_loss=20.34, generator_kl_loss=1.874, generator_dur_loss=1.48, generator_adv_loss=2.241, generator_feat_match_loss=4.122, over 1693.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:11:07,503 INFO [train.py:845] (2/4) Start epoch 573
2024-02-23 19:14:27,076 INFO [train.py:471] (2/4) Epoch 573, batch 36, global_batch_idx: 21200, batch size: 154, loss[discriminator_loss=2.742, discriminator_real_loss=1.513, discriminator_fake_loss=1.23, generator_loss=30.72, generator_mel_loss=20.23, generator_kl_loss=1.791, generator_dur_loss=1.471, generator_adv_loss=2.613, generator_feat_match_loss=4.609, over 154.00 samples.], tot_loss[discriminator_loss=2.568, discriminator_real_loss=1.291, discriminator_fake_loss=1.278, generator_loss=30.18, generator_mel_loss=20.25, generator_kl_loss=1.917, generator_dur_loss=1.481, generator_adv_loss=2.267, generator_feat_match_loss=4.274, over 2768.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 16.0
2024-02-23 19:14:27,078 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 19:14:36,146 INFO [train.py:534] (2/4) Epoch 573, validation: discriminator_loss=2.425, discriminator_real_loss=1.277, discriminator_fake_loss=1.148, generator_loss=31.77, generator_mel_loss=20.97, generator_kl_loss=2.009, generator_dur_loss=1.486, generator_adv_loss=2.621, generator_feat_match_loss=4.691, over 100.00 samples.
2024-02-23 19:14:36,146 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 19:14:36,534 INFO [train.py:845] (2/4) Start epoch 574
2024-02-23 19:18:04,823 INFO [train.py:845] (2/4) Start epoch 575
2024-02-23 19:19:25,798 INFO [train.py:471] (2/4) Epoch 575, batch 12, global_batch_idx: 21250, batch size: 56, loss[discriminator_loss=2.549, discriminator_real_loss=1.282, discriminator_fake_loss=1.267, generator_loss=29.29, generator_mel_loss=19.73, generator_kl_loss=1.859, generator_dur_loss=1.486, generator_adv_loss=2.219, generator_feat_match_loss=3.992, over 56.00 samples.], tot_loss[discriminator_loss=2.619, discriminator_real_loss=1.339, discriminator_fake_loss=1.28, generator_loss=29.55, generator_mel_loss=20.17, generator_kl_loss=1.879, generator_dur_loss=1.486, generator_adv_loss=2.108, generator_feat_match_loss=3.904, over 847.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:21:33,911 INFO [train.py:845] (2/4) Start epoch 576
2024-02-23 19:24:01,173 INFO [train.py:471] (2/4) Epoch 576, batch 25, global_batch_idx: 21300, batch size: 85, loss[discriminator_loss=2.457, discriminator_real_loss=1.281, discriminator_fake_loss=1.176, generator_loss=30.89, generator_mel_loss=20.49, generator_kl_loss=1.803, generator_dur_loss=1.478, generator_adv_loss=2.43, generator_feat_match_loss=4.695, over 85.00 samples.], tot_loss[discriminator_loss=2.561, discriminator_real_loss=1.292, discriminator_fake_loss=1.269, generator_loss=30.16, generator_mel_loss=20.19, generator_kl_loss=1.904, generator_dur_loss=1.48, generator_adv_loss=2.291, generator_feat_match_loss=4.297, over 1837.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:25:07,416 INFO [train.py:845] (2/4) Start epoch 577
2024-02-23 19:28:30,462 INFO [train.py:845] (2/4) Start epoch 578
2024-02-23 19:28:51,111 INFO [train.py:471] (2/4) Epoch 578, batch 1, global_batch_idx: 21350, batch size: 67, loss[discriminator_loss=2.461, discriminator_real_loss=1.316, discriminator_fake_loss=1.145, generator_loss=31.11, generator_mel_loss=20.36, generator_kl_loss=1.827, generator_dur_loss=1.493, generator_adv_loss=2.561, generator_feat_match_loss=4.863, over 67.00 samples.], tot_loss[discriminator_loss=2.58, discriminator_real_loss=1.418, discriminator_fake_loss=1.162, generator_loss=30.52, generator_mel_loss=20.04, generator_kl_loss=1.852, generator_dur_loss=1.508, generator_adv_loss=2.497, generator_feat_match_loss=4.626, over 122.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:32:00,928 INFO [train.py:845] (2/4) Start epoch 579
2024-02-23 19:33:34,752 INFO [train.py:471] (2/4) Epoch 579, batch 14, global_batch_idx: 21400, batch size: 90, loss[discriminator_loss=2.531, discriminator_real_loss=1.298, discriminator_fake_loss=1.234, generator_loss=30.66, generator_mel_loss=20.39, generator_kl_loss=1.952, generator_dur_loss=1.469, generator_adv_loss=2.496, generator_feat_match_loss=4.359, over 90.00 samples.], tot_loss[discriminator_loss=2.583, discriminator_real_loss=1.31, discriminator_fake_loss=1.274, generator_loss=30.26, generator_mel_loss=20.42, generator_kl_loss=1.93, generator_dur_loss=1.482, generator_adv_loss=2.206, generator_feat_match_loss=4.221, over 1115.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:33:34,754 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 19:33:42,644 INFO [train.py:534] (2/4) Epoch 579, validation: discriminator_loss=2.597, discriminator_real_loss=1.286, discriminator_fake_loss=1.311, generator_loss=30.01, generator_mel_loss=20.64, generator_kl_loss=2.079, generator_dur_loss=1.475, generator_adv_loss=1.972, generator_feat_match_loss=3.846, over 100.00 samples.
2024-02-23 19:33:42,645 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 19:35:40,848 INFO [train.py:845] (2/4) Start epoch 580
2024-02-23 19:38:13,278 INFO [train.py:471] (2/4) Epoch 580, batch 27, global_batch_idx: 21450, batch size: 55, loss[discriminator_loss=2.617, discriminator_real_loss=1.318, discriminator_fake_loss=1.3, generator_loss=29.55, generator_mel_loss=20.28, generator_kl_loss=1.93, generator_dur_loss=1.525, generator_adv_loss=2.031, generator_feat_match_loss=3.783, over 55.00 samples.], tot_loss[discriminator_loss=2.534, discriminator_real_loss=1.274, discriminator_fake_loss=1.26, generator_loss=29.99, generator_mel_loss=20.08, generator_kl_loss=1.9, generator_dur_loss=1.483, generator_adv_loss=2.257, generator_feat_match_loss=4.272, over 1994.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:39:05,835 INFO [train.py:845] (2/4) Start epoch 581
2024-02-23 19:42:32,522 INFO [train.py:845] (2/4) Start epoch 582
2024-02-23 19:43:06,746 INFO [train.py:471] (2/4) Epoch 582, batch 3, global_batch_idx: 21500, batch size: 61, loss[discriminator_loss=2.348, discriminator_real_loss=1.157, discriminator_fake_loss=1.191, generator_loss=30.27, generator_mel_loss=20.1, generator_kl_loss=1.811, generator_dur_loss=1.472, generator_adv_loss=2.336, generator_feat_match_loss=4.555, over 61.00 samples.], tot_loss[discriminator_loss=2.52, discriminator_real_loss=1.271, discriminator_fake_loss=1.251, generator_loss=29.9, generator_mel_loss=19.93, generator_kl_loss=1.885, generator_dur_loss=1.471, generator_adv_loss=2.2, generator_feat_match_loss=4.412, over 287.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:46:02,076 INFO [train.py:845] (2/4) Start epoch 583
2024-02-23 19:47:42,003 INFO [train.py:471] (2/4) Epoch 583, batch 16, global_batch_idx: 21550, batch size: 101, loss[discriminator_loss=2.703, discriminator_real_loss=1.315, discriminator_fake_loss=1.388, generator_loss=29.97, generator_mel_loss=20.32, generator_kl_loss=1.841, generator_dur_loss=1.465, generator_adv_loss=2.09, generator_feat_match_loss=4.254, over 101.00 samples.], tot_loss[discriminator_loss=2.578, discriminator_real_loss=1.303, discriminator_fake_loss=1.276, generator_loss=30.38, generator_mel_loss=20.38, generator_kl_loss=1.891, generator_dur_loss=1.479, generator_adv_loss=2.303, generator_feat_match_loss=4.327, over 1311.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:49:31,440 INFO [train.py:845] (2/4) Start epoch 584
2024-02-23 19:52:15,099 INFO [train.py:471] (2/4) Epoch 584, batch 29, global_batch_idx: 21600, batch size: 126, loss[discriminator_loss=2.664, discriminator_real_loss=1.538, discriminator_fake_loss=1.127, generator_loss=30.74, generator_mel_loss=20.69, generator_kl_loss=1.877, generator_dur_loss=1.466, generator_adv_loss=2.506, generator_feat_match_loss=4.199, over 126.00 samples.], tot_loss[discriminator_loss=2.635, discriminator_real_loss=1.349, discriminator_fake_loss=1.286, generator_loss=29.8, generator_mel_loss=20.01, generator_kl_loss=1.924, generator_dur_loss=1.481, generator_adv_loss=2.247, generator_feat_match_loss=4.139, over 2255.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 16.0
2024-02-23 19:52:15,101 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 19:52:23,403 INFO [train.py:534] (2/4) Epoch 584, validation: discriminator_loss=2.624, discriminator_real_loss=1.448, discriminator_fake_loss=1.176, generator_loss=30.84, generator_mel_loss=20.86, generator_kl_loss=2.021, generator_dur_loss=1.481, generator_adv_loss=2.32, generator_feat_match_loss=4.16, over 100.00 samples.
2024-02-23 19:52:23,404 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 19:53:04,525 INFO [train.py:845] (2/4) Start epoch 585
2024-02-23 19:56:32,871 INFO [train.py:845] (2/4) Start epoch 586
2024-02-23 19:57:13,896 INFO [train.py:471] (2/4) Epoch 586, batch 5, global_batch_idx: 21650, batch size: 101, loss[discriminator_loss=3.309, discriminator_real_loss=1.854, discriminator_fake_loss=1.453, generator_loss=28.74, generator_mel_loss=19.85, generator_kl_loss=1.95, generator_dur_loss=1.46, generator_adv_loss=1.898, generator_feat_match_loss=3.578, over 101.00 samples.], tot_loss[discriminator_loss=2.867, discriminator_real_loss=1.477, discriminator_fake_loss=1.391, generator_loss=29.39, generator_mel_loss=19.93, generator_kl_loss=1.874, generator_dur_loss=1.474, generator_adv_loss=2.141, generator_feat_match_loss=3.973, over 398.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 19:59:57,306 INFO [train.py:845] (2/4) Start epoch 587
2024-02-23 20:01:42,107 INFO [train.py:471] (2/4) Epoch 587, batch 18, global_batch_idx: 21700, batch size: 71, loss[discriminator_loss=2.744, discriminator_real_loss=1.404, discriminator_fake_loss=1.34, generator_loss=28.92, generator_mel_loss=19.97, generator_kl_loss=1.961, generator_dur_loss=1.471, generator_adv_loss=1.973, generator_feat_match_loss=3.547, over 71.00 samples.], tot_loss[discriminator_loss=2.599, discriminator_real_loss=1.32, discriminator_fake_loss=1.279, generator_loss=29.78, generator_mel_loss=20.2, generator_kl_loss=1.922, generator_dur_loss=1.477, generator_adv_loss=2.15, generator_feat_match_loss=4.028, over 1398.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:03:24,503 INFO [train.py:845] (2/4) Start epoch 588
2024-02-23 20:06:21,764 INFO [train.py:471] (2/4) Epoch 588, batch 31, global_batch_idx: 21750, batch size: 85, loss[discriminator_loss=2.492, discriminator_real_loss=1.289, discriminator_fake_loss=1.203, generator_loss=30.05, generator_mel_loss=19.82, generator_kl_loss=1.951, generator_dur_loss=1.479, generator_adv_loss=2.385, generator_feat_match_loss=4.414, over 85.00 samples.], tot_loss[discriminator_loss=2.629, discriminator_real_loss=1.329, discriminator_fake_loss=1.299, generator_loss=30.03, generator_mel_loss=20.32, generator_kl_loss=1.919, generator_dur_loss=1.485, generator_adv_loss=2.207, generator_feat_match_loss=4.109, over 2232.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:06:53,961 INFO [train.py:845] (2/4) Start epoch 589
2024-02-23 20:10:24,352 INFO [train.py:845] (2/4) Start epoch 590
2024-02-23 20:11:14,459 INFO [train.py:471] (2/4) Epoch 590, batch 7, global_batch_idx: 21800, batch size: 110, loss[discriminator_loss=2.584, discriminator_real_loss=1.224, discriminator_fake_loss=1.36, generator_loss=29.6, generator_mel_loss=20.31, generator_kl_loss=1.887, generator_dur_loss=1.5, generator_adv_loss=1.974, generator_feat_match_loss=3.93, over 110.00 samples.], tot_loss[discriminator_loss=2.61, discriminator_real_loss=1.312, discriminator_fake_loss=1.299, generator_loss=29.69, generator_mel_loss=20.22, generator_kl_loss=1.881, generator_dur_loss=1.49, generator_adv_loss=2.119, generator_feat_match_loss=3.98, over 550.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:11:14,461 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 20:11:23,480 INFO [train.py:534] (2/4) Epoch 590, validation: discriminator_loss=2.511, discriminator_real_loss=1.132, discriminator_fake_loss=1.379, generator_loss=30.53, generator_mel_loss=21.17, generator_kl_loss=1.96, generator_dur_loss=1.477, generator_adv_loss=1.949, generator_feat_match_loss=3.97, over 100.00 samples.
2024-02-23 20:11:23,481 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 20:14:02,059 INFO [train.py:845] (2/4) Start epoch 591
2024-02-23 20:16:06,961 INFO [train.py:471] (2/4) Epoch 591, batch 20, global_batch_idx: 21850, batch size: 153, loss[discriminator_loss=2.547, discriminator_real_loss=1.217, discriminator_fake_loss=1.329, generator_loss=29.99, generator_mel_loss=20.39, generator_kl_loss=1.822, generator_dur_loss=1.442, generator_adv_loss=2.162, generator_feat_match_loss=4.176, over 153.00 samples.], tot_loss[discriminator_loss=2.577, discriminator_real_loss=1.295, discriminator_fake_loss=1.281, generator_loss=30.28, generator_mel_loss=20.32, generator_kl_loss=1.885, generator_dur_loss=1.476, generator_adv_loss=2.272, generator_feat_match_loss=4.323, over 1609.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:17:31,021 INFO [train.py:845] (2/4) Start epoch 592
2024-02-23 20:20:45,602 INFO [train.py:471] (2/4) Epoch 592, batch 33, global_batch_idx: 21900, batch size: 52, loss[discriminator_loss=2.336, discriminator_real_loss=1.167, discriminator_fake_loss=1.168, generator_loss=30.38, generator_mel_loss=19.91, generator_kl_loss=1.843, generator_dur_loss=1.504, generator_adv_loss=2.324, generator_feat_match_loss=4.797, over 52.00 samples.], tot_loss[discriminator_loss=2.561, discriminator_real_loss=1.313, discriminator_fake_loss=1.248, generator_loss=30.34, generator_mel_loss=20.16, generator_kl_loss=1.92, generator_dur_loss=1.476, generator_adv_loss=2.332, generator_feat_match_loss=4.456, over 2500.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:21:02,044 INFO [train.py:845] (2/4) Start epoch 593
2024-02-23 20:24:32,419 INFO [train.py:845] (2/4) Start epoch 594
2024-02-23 20:25:37,427 INFO [train.py:471] (2/4) Epoch 594, batch 9, global_batch_idx: 21950, batch size: 50, loss[discriminator_loss=2.656, discriminator_real_loss=1.323, discriminator_fake_loss=1.333, generator_loss=29.91, generator_mel_loss=20.03, generator_kl_loss=1.95, generator_dur_loss=1.491, generator_adv_loss=2.363, generator_feat_match_loss=4.07, over 50.00 samples.], tot_loss[discriminator_loss=2.668, discriminator_real_loss=1.338, discriminator_fake_loss=1.33, generator_loss=29.81, generator_mel_loss=20.43, generator_kl_loss=1.899, generator_dur_loss=1.485, generator_adv_loss=2.116, generator_feat_match_loss=3.885, over 644.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:27:58,927 INFO [train.py:845] (2/4) Start epoch 595
2024-02-23 20:30:11,449 INFO [train.py:471] (2/4) Epoch 595, batch 22, global_batch_idx: 22000, batch size: 55, loss[discriminator_loss=2.461, discriminator_real_loss=1.175, discriminator_fake_loss=1.286, generator_loss=30.71, generator_mel_loss=20.21, generator_kl_loss=1.987, generator_dur_loss=1.519, generator_adv_loss=2.07, generator_feat_match_loss=4.922, over 55.00 samples.], tot_loss[discriminator_loss=2.578, discriminator_real_loss=1.31, discriminator_fake_loss=1.268, generator_loss=30.05, generator_mel_loss=20.26, generator_kl_loss=1.916, generator_dur_loss=1.48, generator_adv_loss=2.202, generator_feat_match_loss=4.193, over 1792.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 16.0
2024-02-23 20:30:11,451 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 20:30:19,781 INFO [train.py:534] (2/4) Epoch 595, validation: discriminator_loss=2.672, discriminator_real_loss=1.005, discriminator_fake_loss=1.666, generator_loss=30.49, generator_mel_loss=21.32, generator_kl_loss=2.019, generator_dur_loss=1.482, generator_adv_loss=1.543, generator_feat_match_loss=4.124, over 100.00 samples.
2024-02-23 20:30:19,782 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 20:31:32,453 INFO [train.py:845] (2/4) Start epoch 596
2024-02-23 20:34:55,003 INFO [train.py:471] (2/4) Epoch 596, batch 35, global_batch_idx: 22050, batch size: 76, loss[discriminator_loss=2.551, discriminator_real_loss=1.15, discriminator_fake_loss=1.399, generator_loss=29.75, generator_mel_loss=19.88, generator_kl_loss=1.921, generator_dur_loss=1.495, generator_adv_loss=2.072, generator_feat_match_loss=4.391, over 76.00 samples.], tot_loss[discriminator_loss=2.567, discriminator_real_loss=1.305, discriminator_fake_loss=1.262, generator_loss=30.24, generator_mel_loss=20.12, generator_kl_loss=1.921, generator_dur_loss=1.481, generator_adv_loss=2.31, generator_feat_match_loss=4.407, over 2639.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:34:59,737 INFO [train.py:845] (2/4) Start epoch 597
2024-02-23 20:38:27,995 INFO [train.py:845] (2/4) Start epoch 598
2024-02-23 20:39:43,130 INFO [train.py:471] (2/4) Epoch 598, batch 11, global_batch_idx: 22100, batch size: 59, loss[discriminator_loss=2.301, discriminator_real_loss=1.129, discriminator_fake_loss=1.171, generator_loss=31.05, generator_mel_loss=20.18, generator_kl_loss=1.949, generator_dur_loss=1.507, generator_adv_loss=2.4, generator_feat_match_loss=5.012, over 59.00 samples.], tot_loss[discriminator_loss=2.616, discriminator_real_loss=1.378, discriminator_fake_loss=1.237, generator_loss=30.17, generator_mel_loss=20.03, generator_kl_loss=1.892, generator_dur_loss=1.482, generator_adv_loss=2.302, generator_feat_match_loss=4.461, over 758.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:41:59,165 INFO [train.py:845] (2/4) Start epoch 599
2024-02-23 20:44:16,187 INFO [train.py:471] (2/4) Epoch 599, batch 24, global_batch_idx: 22150, batch size: 61, loss[discriminator_loss=2.484, discriminator_real_loss=1.266, discriminator_fake_loss=1.219, generator_loss=30.42, generator_mel_loss=20.3, generator_kl_loss=1.842, generator_dur_loss=1.465, generator_adv_loss=2.32, generator_feat_match_loss=4.492, over 61.00 samples.], tot_loss[discriminator_loss=2.617, discriminator_real_loss=1.305, discriminator_fake_loss=1.312, generator_loss=29.67, generator_mel_loss=20.14, generator_kl_loss=1.904, generator_dur_loss=1.479, generator_adv_loss=2.11, generator_feat_match_loss=4.039, over 1691.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:45:29,622 INFO [train.py:845] (2/4) Start epoch 600
2024-02-23 20:49:02,012 INFO [train.py:845] (2/4) Start epoch 601
2024-02-23 20:49:14,943 INFO [train.py:471] (2/4) Epoch 601, batch 0, global_batch_idx: 22200, batch size: 58, loss[discriminator_loss=2.359, discriminator_real_loss=1.206, discriminator_fake_loss=1.154, generator_loss=30.55, generator_mel_loss=19.73, generator_kl_loss=1.9, generator_dur_loss=1.511, generator_adv_loss=2.416, generator_feat_match_loss=4.992, over 58.00 samples.], tot_loss[discriminator_loss=2.359, discriminator_real_loss=1.206, discriminator_fake_loss=1.154, generator_loss=30.55, generator_mel_loss=19.73, generator_kl_loss=1.9, generator_dur_loss=1.511, generator_adv_loss=2.416, generator_feat_match_loss=4.992, over 58.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:49:14,945 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 20:49:23,929 INFO [train.py:534] (2/4) Epoch 601, validation: discriminator_loss=2.249, discriminator_real_loss=1.067, discriminator_fake_loss=1.181, generator_loss=31.28, generator_mel_loss=20.49, generator_kl_loss=2.151, generator_dur_loss=1.478, generator_adv_loss=2.184, generator_feat_match_loss=4.975, over 100.00 samples.
2024-02-23 20:49:23,930 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 20:52:42,436 INFO [train.py:845] (2/4) Start epoch 602
2024-02-23 20:54:07,425 INFO [train.py:471] (2/4) Epoch 602, batch 13, global_batch_idx: 22250, batch size: 71, loss[discriminator_loss=2.645, discriminator_real_loss=1.245, discriminator_fake_loss=1.4, generator_loss=29.71, generator_mel_loss=20.29, generator_kl_loss=1.866, generator_dur_loss=1.461, generator_adv_loss=2.223, generator_feat_match_loss=3.875, over 71.00 samples.], tot_loss[discriminator_loss=2.605, discriminator_real_loss=1.306, discriminator_fake_loss=1.299, generator_loss=29.73, generator_mel_loss=20.23, generator_kl_loss=1.928, generator_dur_loss=1.479, generator_adv_loss=2.137, generator_feat_match_loss=3.964, over 1004.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:56:12,086 INFO [train.py:845] (2/4) Start epoch 603
2024-02-23 20:58:45,494 INFO [train.py:471] (2/4) Epoch 603, batch 26, global_batch_idx: 22300, batch size: 49, loss[discriminator_loss=2.6, discriminator_real_loss=1.233, discriminator_fake_loss=1.366, generator_loss=29.03, generator_mel_loss=19.78, generator_kl_loss=1.858, generator_dur_loss=1.493, generator_adv_loss=2.031, generator_feat_match_loss=3.875, over 49.00 samples.], tot_loss[discriminator_loss=2.582, discriminator_real_loss=1.305, discriminator_fake_loss=1.278, generator_loss=30.3, generator_mel_loss=20.3, generator_kl_loss=1.911, generator_dur_loss=1.474, generator_adv_loss=2.3, generator_feat_match_loss=4.314, over 2066.00 samples.], cur_lr_g: 1.86e-04, cur_lr_d: 1.86e-04, grad_scale: 8.0
2024-02-23 20:59:44,187 INFO [train.py:845] (2/4) Start epoch 604
2024-02-23 21:03:11,982 INFO [train.py:845] (2/4) Start epoch 605
2024-02-23 21:03:34,477 INFO [train.py:471] (2/4) Epoch 605, batch 2, global_batch_idx: 22350, batch size: 71, loss[discriminator_loss=2.527, discriminator_real_loss=1.376, discriminator_fake_loss=1.15, generator_loss=30.53, generator_mel_loss=20.04, generator_kl_loss=1.897, generator_dur_loss=1.471, generator_adv_loss=2.383, generator_feat_match_loss=4.742, over 71.00 samples.], tot_loss[discriminator_loss=2.694, discriminator_real_loss=1.399, discriminator_fake_loss=1.296, generator_loss=29.6, generator_mel_loss=19.9, generator_kl_loss=1.905, generator_dur_loss=1.476, generator_adv_loss=2.193, generator_feat_match_loss=4.127, over 172.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:06:41,175 INFO [train.py:845] (2/4) Start epoch 606
2024-02-23 21:08:15,121 INFO [train.py:471] (2/4) Epoch 606, batch 15, global_batch_idx: 22400, batch size: 67, loss[discriminator_loss=2.582, discriminator_real_loss=1.235, discriminator_fake_loss=1.346, generator_loss=30.12, generator_mel_loss=20.27, generator_kl_loss=1.888, generator_dur_loss=1.488, generator_adv_loss=2.168, generator_feat_match_loss=4.305, over 67.00 samples.], tot_loss[discriminator_loss=2.624, discriminator_real_loss=1.326, discriminator_fake_loss=1.297, generator_loss=29.96, generator_mel_loss=20.08, generator_kl_loss=1.926, generator_dur_loss=1.474, generator_adv_loss=2.234, generator_feat_match_loss=4.245, over 1253.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 16.0
2024-02-23 21:08:15,122 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 21:08:24,431 INFO [train.py:534] (2/4) Epoch 606, validation: discriminator_loss=2.667, discriminator_real_loss=1.265, discriminator_fake_loss=1.403, generator_loss=30.11, generator_mel_loss=20.68, generator_kl_loss=2.129, generator_dur_loss=1.474, generator_adv_loss=1.816, generator_feat_match_loss=4.011, over 100.00 samples.
2024-02-23 21:08:24,432 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 21:10:21,186 INFO [train.py:845] (2/4) Start epoch 607
2024-02-23 21:13:10,587 INFO [train.py:471] (2/4) Epoch 607, batch 28, global_batch_idx: 22450, batch size: 76, loss[discriminator_loss=2.971, discriminator_real_loss=1.457, discriminator_fake_loss=1.514, generator_loss=28.58, generator_mel_loss=19.65, generator_kl_loss=1.963, generator_dur_loss=1.466, generator_adv_loss=2.062, generator_feat_match_loss=3.438, over 76.00 samples.], tot_loss[discriminator_loss=2.625, discriminator_real_loss=1.345, discriminator_fake_loss=1.28, generator_loss=30.05, generator_mel_loss=20.17, generator_kl_loss=1.913, generator_dur_loss=1.472, generator_adv_loss=2.227, generator_feat_match_loss=4.27, over 2285.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:13:47,247 INFO [train.py:845] (2/4) Start epoch 608
2024-02-23 21:17:15,429 INFO [train.py:845] (2/4) Start epoch 609
2024-02-23 21:17:49,850 INFO [train.py:471] (2/4) Epoch 609, batch 4, global_batch_idx: 22500, batch size: 153, loss[discriminator_loss=2.562, discriminator_real_loss=1.271, discriminator_fake_loss=1.292, generator_loss=30.21, generator_mel_loss=20.49, generator_kl_loss=1.819, generator_dur_loss=1.451, generator_adv_loss=2.082, generator_feat_match_loss=4.367, over 153.00 samples.], tot_loss[discriminator_loss=2.597, discriminator_real_loss=1.319, discriminator_fake_loss=1.278, generator_loss=29.94, generator_mel_loss=20.27, generator_kl_loss=1.872, generator_dur_loss=1.474, generator_adv_loss=2.157, generator_feat_match_loss=4.163, over 380.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:20:47,907 INFO [train.py:845] (2/4) Start epoch 610
2024-02-23 21:22:31,197 INFO [train.py:471] (2/4) Epoch 610, batch 17, global_batch_idx: 22550, batch size: 52, loss[discriminator_loss=2.871, discriminator_real_loss=1.322, discriminator_fake_loss=1.549, generator_loss=29.32, generator_mel_loss=20.32, generator_kl_loss=1.916, generator_dur_loss=1.502, generator_adv_loss=1.915, generator_feat_match_loss=3.672, over 52.00 samples.], tot_loss[discriminator_loss=2.555, discriminator_real_loss=1.285, discriminator_fake_loss=1.271, generator_loss=30.28, generator_mel_loss=20.25, generator_kl_loss=1.931, generator_dur_loss=1.482, generator_adv_loss=2.264, generator_feat_match_loss=4.358, over 1317.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:24:21,315 INFO [train.py:845] (2/4) Start epoch 611
2024-02-23 21:27:24,340 INFO [train.py:471] (2/4) Epoch 611, batch 30, global_batch_idx: 22600, batch size: 73, loss[discriminator_loss=2.551, discriminator_real_loss=1.285, discriminator_fake_loss=1.267, generator_loss=30.31, generator_mel_loss=20.34, generator_kl_loss=1.863, generator_dur_loss=1.454, generator_adv_loss=2.215, generator_feat_match_loss=4.438, over 73.00 samples.], tot_loss[discriminator_loss=2.52, discriminator_real_loss=1.271, discriminator_fake_loss=1.249, generator_loss=30.55, generator_mel_loss=20.17, generator_kl_loss=1.919, generator_dur_loss=1.478, generator_adv_loss=2.372, generator_feat_match_loss=4.617, over 2245.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:27:24,341 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 21:27:32,607 INFO [train.py:534] (2/4) Epoch 611, validation: discriminator_loss=2.492, discriminator_real_loss=1.13, discriminator_fake_loss=1.363, generator_loss=30.45, generator_mel_loss=20.73, generator_kl_loss=2.005, generator_dur_loss=1.475, generator_adv_loss=1.961, generator_feat_match_loss=4.283, over 100.00 samples.
2024-02-23 21:27:32,607 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 21:27:58,809 INFO [train.py:845] (2/4) Start epoch 612
2024-02-23 21:31:26,220 INFO [train.py:845] (2/4) Start epoch 613
2024-02-23 21:32:09,507 INFO [train.py:471] (2/4) Epoch 613, batch 6, global_batch_idx: 22650, batch size: 67, loss[discriminator_loss=2.4, discriminator_real_loss=1.182, discriminator_fake_loss=1.219, generator_loss=29.6, generator_mel_loss=19.53, generator_kl_loss=1.846, generator_dur_loss=1.475, generator_adv_loss=2.301, generator_feat_match_loss=4.445, over 67.00 samples.], tot_loss[discriminator_loss=2.742, discriminator_real_loss=1.372, discriminator_fake_loss=1.37, generator_loss=29.62, generator_mel_loss=19.94, generator_kl_loss=1.907, generator_dur_loss=1.48, generator_adv_loss=2.127, generator_feat_match_loss=4.167, over 436.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:34:54,490 INFO [train.py:845] (2/4) Start epoch 614
2024-02-23 21:36:56,370 INFO [train.py:471] (2/4) Epoch 614, batch 19, global_batch_idx: 22700, batch size: 153, loss[discriminator_loss=2.34, discriminator_real_loss=1.196, discriminator_fake_loss=1.144, generator_loss=31.02, generator_mel_loss=20.34, generator_kl_loss=1.927, generator_dur_loss=1.449, generator_adv_loss=2.303, generator_feat_match_loss=5, over 153.00 samples.], tot_loss[discriminator_loss=2.546, discriminator_real_loss=1.309, discriminator_fake_loss=1.237, generator_loss=30.33, generator_mel_loss=20.29, generator_kl_loss=1.897, generator_dur_loss=1.472, generator_adv_loss=2.263, generator_feat_match_loss=4.404, over 1464.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:38:24,349 INFO [train.py:845] (2/4) Start epoch 615
2024-02-23 21:41:27,998 INFO [train.py:471] (2/4) Epoch 615, batch 32, global_batch_idx: 22750, batch size: 52, loss[discriminator_loss=2.496, discriminator_real_loss=1.274, discriminator_fake_loss=1.223, generator_loss=30.34, generator_mel_loss=19.91, generator_kl_loss=1.816, generator_dur_loss=1.494, generator_adv_loss=2.434, generator_feat_match_loss=4.688, over 52.00 samples.], tot_loss[discriminator_loss=2.6, discriminator_real_loss=1.32, discriminator_fake_loss=1.281, generator_loss=30.26, generator_mel_loss=20.2, generator_kl_loss=1.897, generator_dur_loss=1.477, generator_adv_loss=2.296, generator_feat_match_loss=4.389, over 2338.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:41:52,218 INFO [train.py:845] (2/4) Start epoch 616
2024-02-23 21:45:22,669 INFO [train.py:845] (2/4) Start epoch 617
2024-02-23 21:46:17,246 INFO [train.py:471] (2/4) Epoch 617, batch 8, global_batch_idx: 22800, batch size: 76, loss[discriminator_loss=2.605, discriminator_real_loss=1.396, discriminator_fake_loss=1.209, generator_loss=28.98, generator_mel_loss=19.66, generator_kl_loss=1.909, generator_dur_loss=1.499, generator_adv_loss=2.025, generator_feat_match_loss=3.891, over 76.00 samples.], tot_loss[discriminator_loss=2.608, discriminator_real_loss=1.342, discriminator_fake_loss=1.266, generator_loss=29.75, generator_mel_loss=20.12, generator_kl_loss=1.924, generator_dur_loss=1.487, generator_adv_loss=2.129, generator_feat_match_loss=4.087, over 665.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 16.0
2024-02-23 21:46:17,247 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 21:46:25,931 INFO [train.py:534] (2/4) Epoch 617, validation: discriminator_loss=2.678, discriminator_real_loss=1.22, discriminator_fake_loss=1.458, generator_loss=29.99, generator_mel_loss=20.75, generator_kl_loss=2.099, generator_dur_loss=1.477, generator_adv_loss=1.742, generator_feat_match_loss=3.918, over 100.00 samples.
2024-02-23 21:46:25,932 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 21:49:06,428 INFO [train.py:845] (2/4) Start epoch 618
2024-02-23 21:51:12,106 INFO [train.py:471] (2/4) Epoch 618, batch 21, global_batch_idx: 22850, batch size: 110, loss[discriminator_loss=2.559, discriminator_real_loss=1.252, discriminator_fake_loss=1.307, generator_loss=30.41, generator_mel_loss=20.38, generator_kl_loss=1.998, generator_dur_loss=1.461, generator_adv_loss=2.242, generator_feat_match_loss=4.324, over 110.00 samples.], tot_loss[discriminator_loss=2.57, discriminator_real_loss=1.299, discriminator_fake_loss=1.271, generator_loss=30.26, generator_mel_loss=20.26, generator_kl_loss=1.934, generator_dur_loss=1.478, generator_adv_loss=2.245, generator_feat_match_loss=4.344, over 1692.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:52:31,931 INFO [train.py:845] (2/4) Start epoch 619
2024-02-23 21:55:55,972 INFO [train.py:471] (2/4) Epoch 619, batch 34, global_batch_idx: 22900, batch size: 60, loss[discriminator_loss=2.564, discriminator_real_loss=1.203, discriminator_fake_loss=1.361, generator_loss=29.52, generator_mel_loss=19.72, generator_kl_loss=1.866, generator_dur_loss=1.481, generator_adv_loss=2.271, generator_feat_match_loss=4.18, over 60.00 samples.], tot_loss[discriminator_loss=2.566, discriminator_real_loss=1.28, discriminator_fake_loss=1.286, generator_loss=30.38, generator_mel_loss=20.17, generator_kl_loss=1.902, generator_dur_loss=1.474, generator_adv_loss=2.298, generator_feat_match_loss=4.54, over 2587.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 21:56:05,393 INFO [train.py:845] (2/4) Start epoch 620
2024-02-23 21:59:38,485 INFO [train.py:845] (2/4) Start epoch 621
2024-02-23 22:00:50,369 INFO [train.py:471] (2/4) Epoch 621, batch 10, global_batch_idx: 22950, batch size: 110, loss[discriminator_loss=2.609, discriminator_real_loss=1.288, discriminator_fake_loss=1.322, generator_loss=30.14, generator_mel_loss=20.47, generator_kl_loss=1.882, generator_dur_loss=1.476, generator_adv_loss=2.234, generator_feat_match_loss=4.078, over 110.00 samples.], tot_loss[discriminator_loss=2.595, discriminator_real_loss=1.307, discriminator_fake_loss=1.288, generator_loss=30.01, generator_mel_loss=20.32, generator_kl_loss=1.963, generator_dur_loss=1.474, generator_adv_loss=2.15, generator_feat_match_loss=4.107, over 846.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:03:05,523 INFO [train.py:845] (2/4) Start epoch 622
2024-02-23 22:05:14,424 INFO [train.py:471] (2/4) Epoch 622, batch 23, global_batch_idx: 23000, batch size: 126, loss[discriminator_loss=2.709, discriminator_real_loss=1.521, discriminator_fake_loss=1.188, generator_loss=29.07, generator_mel_loss=19.99, generator_kl_loss=1.956, generator_dur_loss=1.46, generator_adv_loss=1.92, generator_feat_match_loss=3.744, over 126.00 samples.], tot_loss[discriminator_loss=2.585, discriminator_real_loss=1.312, discriminator_fake_loss=1.274, generator_loss=30.1, generator_mel_loss=20.07, generator_kl_loss=1.926, generator_dur_loss=1.482, generator_adv_loss=2.234, generator_feat_match_loss=4.388, over 1575.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:05:14,426 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 22:05:21,924 INFO [train.py:534] (2/4) Epoch 622, validation: discriminator_loss=2.67, discriminator_real_loss=1.258, discriminator_fake_loss=1.413, generator_loss=29.98, generator_mel_loss=20.83, generator_kl_loss=1.95, generator_dur_loss=1.49, generator_adv_loss=1.909, generator_feat_match_loss=3.807, over 100.00 samples.
2024-02-23 22:05:21,925 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 22:06:41,031 INFO [train.py:845] (2/4) Start epoch 623
2024-02-23 22:10:06,712 INFO [train.py:471] (2/4) Epoch 623, batch 36, global_batch_idx: 23050, batch size: 81, loss[discriminator_loss=2.666, discriminator_real_loss=1.479, discriminator_fake_loss=1.187, generator_loss=30.8, generator_mel_loss=20.57, generator_kl_loss=2.04, generator_dur_loss=1.493, generator_adv_loss=2.35, generator_feat_match_loss=4.352, over 81.00 samples.], tot_loss[discriminator_loss=2.594, discriminator_real_loss=1.302, discriminator_fake_loss=1.292, generator_loss=30.07, generator_mel_loss=20.31, generator_kl_loss=1.93, generator_dur_loss=1.478, generator_adv_loss=2.169, generator_feat_match_loss=4.184, over 2753.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:10:07,169 INFO [train.py:845] (2/4) Start epoch 624
2024-02-23 22:13:30,597 INFO [train.py:845] (2/4) Start epoch 625
2024-02-23 22:14:46,663 INFO [train.py:471] (2/4) Epoch 625, batch 12, global_batch_idx: 23100, batch size: 50, loss[discriminator_loss=2.35, discriminator_real_loss=1.074, discriminator_fake_loss=1.275, generator_loss=30.44, generator_mel_loss=19.92, generator_kl_loss=1.894, generator_dur_loss=1.464, generator_adv_loss=2.324, generator_feat_match_loss=4.836, over 50.00 samples.], tot_loss[discriminator_loss=2.511, discriminator_real_loss=1.272, discriminator_fake_loss=1.239, generator_loss=30.07, generator_mel_loss=19.89, generator_kl_loss=1.874, generator_dur_loss=1.481, generator_adv_loss=2.27, generator_feat_match_loss=4.557, over 929.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:16:59,477 INFO [train.py:845] (2/4) Start epoch 626
2024-02-23 22:19:23,213 INFO [train.py:471] (2/4) Epoch 626, batch 25, global_batch_idx: 23150, batch size: 50, loss[discriminator_loss=2.676, discriminator_real_loss=1.46, discriminator_fake_loss=1.217, generator_loss=30.37, generator_mel_loss=20.53, generator_kl_loss=1.943, generator_dur_loss=1.516, generator_adv_loss=2.268, generator_feat_match_loss=4.109, over 50.00 samples.], tot_loss[discriminator_loss=2.61, discriminator_real_loss=1.316, discriminator_fake_loss=1.295, generator_loss=30.13, generator_mel_loss=20.41, generator_kl_loss=1.93, generator_dur_loss=1.479, generator_adv_loss=2.166, generator_feat_match_loss=4.151, over 1818.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:20:26,419 INFO [train.py:845] (2/4) Start epoch 627
2024-02-23 22:24:00,744 INFO [train.py:845] (2/4) Start epoch 628
2024-02-23 22:24:20,781 INFO [train.py:471] (2/4) Epoch 628, batch 1, global_batch_idx: 23200, batch size: 81, loss[discriminator_loss=2.75, discriminator_real_loss=1.188, discriminator_fake_loss=1.562, generator_loss=29.65, generator_mel_loss=20.23, generator_kl_loss=1.976, generator_dur_loss=1.472, generator_adv_loss=2.021, generator_feat_match_loss=3.949, over 81.00 samples.], tot_loss[discriminator_loss=2.68, discriminator_real_loss=1.162, discriminator_fake_loss=1.519, generator_loss=29.54, generator_mel_loss=20.02, generator_kl_loss=1.93, generator_dur_loss=1.486, generator_adv_loss=2.059, generator_feat_match_loss=4.045, over 135.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 16.0
2024-02-23 22:24:20,783 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 22:24:29,205 INFO [train.py:534] (2/4) Epoch 628, validation: discriminator_loss=2.579, discriminator_real_loss=1.18, discriminator_fake_loss=1.399, generator_loss=30.14, generator_mel_loss=20.74, generator_kl_loss=2.018, generator_dur_loss=1.475, generator_adv_loss=1.915, generator_feat_match_loss=3.996, over 100.00 samples.
2024-02-23 22:24:29,206 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 22:27:33,445 INFO [train.py:845] (2/4) Start epoch 629
2024-02-23 22:29:01,810 INFO [train.py:471] (2/4) Epoch 629, batch 14, global_batch_idx: 23250, batch size: 81, loss[discriminator_loss=2.449, discriminator_real_loss=1.183, discriminator_fake_loss=1.266, generator_loss=30.5, generator_mel_loss=20.31, generator_kl_loss=1.808, generator_dur_loss=1.474, generator_adv_loss=2.369, generator_feat_match_loss=4.535, over 81.00 samples.], tot_loss[discriminator_loss=2.482, discriminator_real_loss=1.242, discriminator_fake_loss=1.239, generator_loss=30.41, generator_mel_loss=20.2, generator_kl_loss=1.914, generator_dur_loss=1.478, generator_adv_loss=2.279, generator_feat_match_loss=4.545, over 1116.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:31:02,267 INFO [train.py:845] (2/4) Start epoch 630
2024-02-23 22:33:46,969 INFO [train.py:471] (2/4) Epoch 630, batch 27, global_batch_idx: 23300, batch size: 76, loss[discriminator_loss=2.344, discriminator_real_loss=1.219, discriminator_fake_loss=1.125, generator_loss=31.1, generator_mel_loss=20.07, generator_kl_loss=1.875, generator_dur_loss=1.5, generator_adv_loss=2.561, generator_feat_match_loss=5.094, over 76.00 samples.], tot_loss[discriminator_loss=2.538, discriminator_real_loss=1.271, discriminator_fake_loss=1.268, generator_loss=30.42, generator_mel_loss=20.09, generator_kl_loss=1.916, generator_dur_loss=1.475, generator_adv_loss=2.348, generator_feat_match_loss=4.592, over 1953.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:34:31,508 INFO [train.py:845] (2/4) Start epoch 631
2024-02-23 22:38:02,195 INFO [train.py:845] (2/4) Start epoch 632
2024-02-23 22:38:29,945 INFO [train.py:471] (2/4) Epoch 632, batch 3, global_batch_idx: 23350, batch size: 58, loss[discriminator_loss=2.434, discriminator_real_loss=1.225, discriminator_fake_loss=1.209, generator_loss=30.63, generator_mel_loss=19.97, generator_kl_loss=1.879, generator_dur_loss=1.472, generator_adv_loss=2.402, generator_feat_match_loss=4.902, over 58.00 samples.], tot_loss[discriminator_loss=2.589, discriminator_real_loss=1.309, discriminator_fake_loss=1.279, generator_loss=30.28, generator_mel_loss=20.19, generator_kl_loss=1.879, generator_dur_loss=1.479, generator_adv_loss=2.257, generator_feat_match_loss=4.48, over 249.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:41:30,019 INFO [train.py:845] (2/4) Start epoch 633
2024-02-23 22:43:14,750 INFO [train.py:471] (2/4) Epoch 633, batch 16, global_batch_idx: 23400, batch size: 49, loss[discriminator_loss=2.666, discriminator_real_loss=1.147, discriminator_fake_loss=1.519, generator_loss=29.48, generator_mel_loss=20.06, generator_kl_loss=1.859, generator_dur_loss=1.487, generator_adv_loss=2.045, generator_feat_match_loss=4.023, over 49.00 samples.], tot_loss[discriminator_loss=2.581, discriminator_real_loss=1.297, discriminator_fake_loss=1.284, generator_loss=29.98, generator_mel_loss=19.98, generator_kl_loss=1.92, generator_dur_loss=1.473, generator_adv_loss=2.228, generator_feat_match_loss=4.38, over 1328.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:43:14,752 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 22:43:23,008 INFO [train.py:534] (2/4) Epoch 633, validation: discriminator_loss=2.503, discriminator_real_loss=1.229, discriminator_fake_loss=1.274, generator_loss=30.52, generator_mel_loss=20.75, generator_kl_loss=2.036, generator_dur_loss=1.476, generator_adv_loss=1.971, generator_feat_match_loss=4.291, over 100.00 samples.
2024-02-23 22:43:23,009 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 22:45:04,089 INFO [train.py:845] (2/4) Start epoch 634
2024-02-23 22:47:51,061 INFO [train.py:471] (2/4) Epoch 634, batch 29, global_batch_idx: 23450, batch size: 90, loss[discriminator_loss=2.711, discriminator_real_loss=1.159, discriminator_fake_loss=1.551, generator_loss=30.38, generator_mel_loss=20.74, generator_kl_loss=1.896, generator_dur_loss=1.483, generator_adv_loss=2.189, generator_feat_match_loss=4.074, over 90.00 samples.], tot_loss[discriminator_loss=2.582, discriminator_real_loss=1.301, discriminator_fake_loss=1.281, generator_loss=30.34, generator_mel_loss=20.37, generator_kl_loss=1.951, generator_dur_loss=1.478, generator_adv_loss=2.223, generator_feat_match_loss=4.32, over 2045.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:48:33,589 INFO [train.py:845] (2/4) Start epoch 635
2024-02-23 22:52:11,429 INFO [train.py:845] (2/4) Start epoch 636
2024-02-23 22:52:48,081 INFO [train.py:471] (2/4) Epoch 636, batch 5, global_batch_idx: 23500, batch size: 71, loss[discriminator_loss=2.578, discriminator_real_loss=1.423, discriminator_fake_loss=1.154, generator_loss=29.68, generator_mel_loss=20.01, generator_kl_loss=1.869, generator_dur_loss=1.484, generator_adv_loss=2.268, generator_feat_match_loss=4.043, over 71.00 samples.], tot_loss[discriminator_loss=2.569, discriminator_real_loss=1.294, discriminator_fake_loss=1.275, generator_loss=30.06, generator_mel_loss=20.09, generator_kl_loss=1.918, generator_dur_loss=1.477, generator_adv_loss=2.253, generator_feat_match_loss=4.324, over 393.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:55:42,971 INFO [train.py:845] (2/4) Start epoch 637
2024-02-23 22:57:38,551 INFO [train.py:471] (2/4) Epoch 637, batch 18, global_batch_idx: 23550, batch size: 49, loss[discriminator_loss=2.695, discriminator_real_loss=1.32, discriminator_fake_loss=1.376, generator_loss=29.32, generator_mel_loss=19.76, generator_kl_loss=1.836, generator_dur_loss=1.504, generator_adv_loss=2.098, generator_feat_match_loss=4.121, over 49.00 samples.], tot_loss[discriminator_loss=2.541, discriminator_real_loss=1.272, discriminator_fake_loss=1.269, generator_loss=30.17, generator_mel_loss=19.99, generator_kl_loss=1.898, generator_dur_loss=1.472, generator_adv_loss=2.291, generator_feat_match_loss=4.522, over 1313.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 22:59:08,170 INFO [train.py:845] (2/4) Start epoch 638
2024-02-23 23:02:09,913 INFO [train.py:471] (2/4) Epoch 638, batch 31, global_batch_idx: 23600, batch size: 61, loss[discriminator_loss=2.641, discriminator_real_loss=1.366, discriminator_fake_loss=1.275, generator_loss=29.98, generator_mel_loss=20.12, generator_kl_loss=1.86, generator_dur_loss=1.507, generator_adv_loss=2.16, generator_feat_match_loss=4.34, over 61.00 samples.], tot_loss[discriminator_loss=2.485, discriminator_real_loss=1.235, discriminator_fake_loss=1.25, generator_loss=30.5, generator_mel_loss=20.03, generator_kl_loss=1.931, generator_dur_loss=1.477, generator_adv_loss=2.342, generator_feat_match_loss=4.723, over 2382.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 16.0
2024-02-23 23:02:09,915 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 23:02:18,497 INFO [train.py:534] (2/4) Epoch 638, validation: discriminator_loss=2.534, discriminator_real_loss=1.228, discriminator_fake_loss=1.306, generator_loss=30.89, generator_mel_loss=20.93, generator_kl_loss=1.999, generator_dur_loss=1.478, generator_adv_loss=2.078, generator_feat_match_loss=4.401, over 100.00 samples.
2024-02-23 23:02:18,498 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 23:02:44,381 INFO [train.py:845] (2/4) Start epoch 639
2024-02-23 23:06:13,904 INFO [train.py:845] (2/4) Start epoch 640
2024-02-23 23:07:06,834 INFO [train.py:471] (2/4) Epoch 640, batch 7, global_batch_idx: 23650, batch size: 90, loss[discriminator_loss=2.559, discriminator_real_loss=1.252, discriminator_fake_loss=1.308, generator_loss=30.19, generator_mel_loss=20.38, generator_kl_loss=1.932, generator_dur_loss=1.458, generator_adv_loss=2.219, generator_feat_match_loss=4.199, over 90.00 samples.], tot_loss[discriminator_loss=2.55, discriminator_real_loss=1.279, discriminator_fake_loss=1.272, generator_loss=30.58, generator_mel_loss=20.4, generator_kl_loss=1.955, generator_dur_loss=1.471, generator_adv_loss=2.311, generator_feat_match_loss=4.443, over 613.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 23:09:37,532 INFO [train.py:845] (2/4) Start epoch 641
2024-02-23 23:11:39,855 INFO [train.py:471] (2/4) Epoch 641, batch 20, global_batch_idx: 23700, batch size: 126, loss[discriminator_loss=2.449, discriminator_real_loss=1.226, discriminator_fake_loss=1.224, generator_loss=31.45, generator_mel_loss=20.16, generator_kl_loss=1.914, generator_dur_loss=1.476, generator_adv_loss=2.613, generator_feat_match_loss=5.289, over 126.00 samples.], tot_loss[discriminator_loss=2.492, discriminator_real_loss=1.247, discriminator_fake_loss=1.245, generator_loss=30.59, generator_mel_loss=20.06, generator_kl_loss=1.91, generator_dur_loss=1.476, generator_adv_loss=2.39, generator_feat_match_loss=4.752, over 1559.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 23:13:06,596 INFO [train.py:845] (2/4) Start epoch 642
2024-02-23 23:16:19,785 INFO [train.py:471] (2/4) Epoch 642, batch 33, global_batch_idx: 23750, batch size: 55, loss[discriminator_loss=2.588, discriminator_real_loss=1.438, discriminator_fake_loss=1.15, generator_loss=29.79, generator_mel_loss=19.96, generator_kl_loss=1.901, generator_dur_loss=1.526, generator_adv_loss=2.371, generator_feat_match_loss=4.031, over 55.00 samples.], tot_loss[discriminator_loss=2.554, discriminator_real_loss=1.285, discriminator_fake_loss=1.269, generator_loss=30.28, generator_mel_loss=20.09, generator_kl_loss=1.903, generator_dur_loss=1.475, generator_adv_loss=2.278, generator_feat_match_loss=4.535, over 2575.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 23:16:34,579 INFO [train.py:845] (2/4) Start epoch 643
2024-02-23 23:20:05,698 INFO [train.py:845] (2/4) Start epoch 644
2024-02-23 23:21:08,319 INFO [train.py:471] (2/4) Epoch 644, batch 9, global_batch_idx: 23800, batch size: 50, loss[discriminator_loss=2.459, discriminator_real_loss=1.258, discriminator_fake_loss=1.201, generator_loss=30.56, generator_mel_loss=19.98, generator_kl_loss=1.848, generator_dur_loss=1.495, generator_adv_loss=2.395, generator_feat_match_loss=4.844, over 50.00 samples.], tot_loss[discriminator_loss=2.535, discriminator_real_loss=1.265, discriminator_fake_loss=1.27, generator_loss=30.22, generator_mel_loss=19.92, generator_kl_loss=1.908, generator_dur_loss=1.477, generator_adv_loss=2.334, generator_feat_match_loss=4.579, over 672.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 23:21:08,320 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 23:21:16,038 INFO [train.py:534] (2/4) Epoch 644, validation: discriminator_loss=2.447, discriminator_real_loss=1.232, discriminator_fake_loss=1.215, generator_loss=30.7, generator_mel_loss=20.56, generator_kl_loss=1.99, generator_dur_loss=1.477, generator_adv_loss=2.103, generator_feat_match_loss=4.574, over 100.00 samples.
2024-02-23 23:21:16,039 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 23:23:45,833 INFO [train.py:845] (2/4) Start epoch 645
2024-02-23 23:25:57,548 INFO [train.py:471] (2/4) Epoch 645, batch 22, global_batch_idx: 23850, batch size: 63, loss[discriminator_loss=2.809, discriminator_real_loss=1.199, discriminator_fake_loss=1.61, generator_loss=28.93, generator_mel_loss=19.65, generator_kl_loss=1.867, generator_dur_loss=1.474, generator_adv_loss=2, generator_feat_match_loss=3.943, over 63.00 samples.], tot_loss[discriminator_loss=2.444, discriminator_real_loss=1.225, discriminator_fake_loss=1.22, generator_loss=30.69, generator_mel_loss=20.01, generator_kl_loss=1.941, generator_dur_loss=1.48, generator_adv_loss=2.402, generator_feat_match_loss=4.864, over 1570.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 23:27:12,476 INFO [train.py:845] (2/4) Start epoch 646
2024-02-23 23:30:39,015 INFO [train.py:471] (2/4) Epoch 646, batch 35, global_batch_idx: 23900, batch size: 63, loss[discriminator_loss=2.465, discriminator_real_loss=1.192, discriminator_fake_loss=1.272, generator_loss=29.93, generator_mel_loss=19.32, generator_kl_loss=1.923, generator_dur_loss=1.491, generator_adv_loss=2.23, generator_feat_match_loss=4.961, over 63.00 samples.], tot_loss[discriminator_loss=2.416, discriminator_real_loss=1.212, discriminator_fake_loss=1.204, generator_loss=30.53, generator_mel_loss=19.8, generator_kl_loss=1.912, generator_dur_loss=1.473, generator_adv_loss=2.427, generator_feat_match_loss=4.917, over 2737.00 samples.], cur_lr_g: 1.85e-04, cur_lr_d: 1.85e-04, grad_scale: 8.0
2024-02-23 23:30:43,811 INFO [train.py:845] (2/4) Start epoch 647
2024-02-23 23:34:06,079 INFO [train.py:845] (2/4) Start epoch 648
2024-02-23 23:35:16,061 INFO [train.py:471] (2/4) Epoch 648, batch 11, global_batch_idx: 23950, batch size: 60, loss[discriminator_loss=2.621, discriminator_real_loss=1.326, discriminator_fake_loss=1.295, generator_loss=30.24, generator_mel_loss=19.64, generator_kl_loss=1.884, generator_dur_loss=1.475, generator_adv_loss=2.299, generator_feat_match_loss=4.945, over 60.00 samples.], tot_loss[discriminator_loss=2.554, discriminator_real_loss=1.28, discriminator_fake_loss=1.273, generator_loss=30.3, generator_mel_loss=19.93, generator_kl_loss=1.905, generator_dur_loss=1.478, generator_adv_loss=2.339, generator_feat_match_loss=4.645, over 849.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-23 23:37:33,663 INFO [train.py:845] (2/4) Start epoch 649
2024-02-23 23:40:00,767 INFO [train.py:471] (2/4) Epoch 649, batch 24, global_batch_idx: 24000, batch size: 52, loss[discriminator_loss=2.539, discriminator_real_loss=1.326, discriminator_fake_loss=1.212, generator_loss=29.63, generator_mel_loss=19.51, generator_kl_loss=1.955, generator_dur_loss=1.496, generator_adv_loss=2.125, generator_feat_match_loss=4.539, over 52.00 samples.], tot_loss[discriminator_loss=2.445, discriminator_real_loss=1.231, discriminator_fake_loss=1.214, generator_loss=30.43, generator_mel_loss=19.79, generator_kl_loss=1.933, generator_dur_loss=1.48, generator_adv_loss=2.405, generator_feat_match_loss=4.819, over 1632.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 16.0
2024-02-23 23:40:00,769 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 23:40:08,792 INFO [train.py:534] (2/4) Epoch 649, validation: discriminator_loss=2.826, discriminator_real_loss=1.135, discriminator_fake_loss=1.69, generator_loss=29.89, generator_mel_loss=20.67, generator_kl_loss=2.152, generator_dur_loss=1.476, generator_adv_loss=1.588, generator_feat_match_loss=4.01, over 100.00 samples.
2024-02-23 23:40:08,793 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-23 23:41:10,809 INFO [train.py:845] (2/4) Start epoch 650
2024-02-23 23:44:40,757 INFO [train.py:845] (2/4) Start epoch 651
2024-02-23 23:44:54,558 INFO [train.py:471] (2/4) Epoch 651, batch 0, global_batch_idx: 24050, batch size: 56, loss[discriminator_loss=2.26, discriminator_real_loss=1.229, discriminator_fake_loss=1.031, generator_loss=30.74, generator_mel_loss=19.6, generator_kl_loss=1.993, generator_dur_loss=1.494, generator_adv_loss=2.529, generator_feat_match_loss=5.121, over 56.00 samples.], tot_loss[discriminator_loss=2.26, discriminator_real_loss=1.229, discriminator_fake_loss=1.031, generator_loss=30.74, generator_mel_loss=19.6, generator_kl_loss=1.993, generator_dur_loss=1.494, generator_adv_loss=2.529, generator_feat_match_loss=5.121, over 56.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-23 23:48:13,727 INFO [train.py:845] (2/4) Start epoch 652
2024-02-23 23:49:46,690 INFO [train.py:471] (2/4) Epoch 652, batch 13, global_batch_idx: 24100, batch size: 56, loss[discriminator_loss=2.602, discriminator_real_loss=1.294, discriminator_fake_loss=1.309, generator_loss=29.71, generator_mel_loss=19.97, generator_kl_loss=1.898, generator_dur_loss=1.464, generator_adv_loss=2.281, generator_feat_match_loss=4.094, over 56.00 samples.], tot_loss[discriminator_loss=2.537, discriminator_real_loss=1.28, discriminator_fake_loss=1.257, generator_loss=30.47, generator_mel_loss=20.17, generator_kl_loss=1.951, generator_dur_loss=1.481, generator_adv_loss=2.299, generator_feat_match_loss=4.577, over 956.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-23 23:51:48,101 INFO [train.py:845] (2/4) Start epoch 653
2024-02-23 23:54:24,181 INFO [train.py:471] (2/4) Epoch 653, batch 26, global_batch_idx: 24150, batch size: 95, loss[discriminator_loss=2.465, discriminator_real_loss=1.253, discriminator_fake_loss=1.211, generator_loss=30.53, generator_mel_loss=20.02, generator_kl_loss=1.959, generator_dur_loss=1.447, generator_adv_loss=2.213, generator_feat_match_loss=4.891, over 95.00 samples.], tot_loss[discriminator_loss=2.465, discriminator_real_loss=1.221, discriminator_fake_loss=1.244, generator_loss=30.43, generator_mel_loss=19.88, generator_kl_loss=1.926, generator_dur_loss=1.47, generator_adv_loss=2.366, generator_feat_match_loss=4.783, over 2202.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-23 23:55:19,842 INFO [train.py:845] (2/4) Start epoch 654
2024-02-23 23:58:43,452 INFO [train.py:845] (2/4) Start epoch 655
2024-02-23 23:59:06,360 INFO [train.py:471] (2/4) Epoch 655, batch 2, global_batch_idx: 24200, batch size: 59, loss[discriminator_loss=2.656, discriminator_real_loss=1.245, discriminator_fake_loss=1.41, generator_loss=30.28, generator_mel_loss=20.21, generator_kl_loss=1.956, generator_dur_loss=1.496, generator_adv_loss=2.121, generator_feat_match_loss=4.5, over 59.00 samples.], tot_loss[discriminator_loss=2.507, discriminator_real_loss=1.256, discriminator_fake_loss=1.25, generator_loss=30.42, generator_mel_loss=19.95, generator_kl_loss=1.908, generator_dur_loss=1.477, generator_adv_loss=2.343, generator_feat_match_loss=4.743, over 186.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-23 23:59:06,360 INFO [train.py:525] (2/4) Computing validation loss
2024-02-23 23:59:15,149 INFO [train.py:534] (2/4) Epoch 655, validation: discriminator_loss=2.603, discriminator_real_loss=1.198, discriminator_fake_loss=1.405, generator_loss=30.21, generator_mel_loss=20.53, generator_kl_loss=1.976, generator_dur_loss=1.479, generator_adv_loss=1.886, generator_feat_match_loss=4.341, over 100.00 samples.
2024-02-23 23:59:15,149 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 00:02:25,356 INFO [train.py:845] (2/4) Start epoch 656
2024-02-24 00:04:02,932 INFO [train.py:471] (2/4) Epoch 656, batch 15, global_batch_idx: 24250, batch size: 64, loss[discriminator_loss=2.395, discriminator_real_loss=1.183, discriminator_fake_loss=1.212, generator_loss=30.97, generator_mel_loss=20.1, generator_kl_loss=1.936, generator_dur_loss=1.494, generator_adv_loss=2.414, generator_feat_match_loss=5.027, over 64.00 samples.], tot_loss[discriminator_loss=2.465, discriminator_real_loss=1.26, discriminator_fake_loss=1.205, generator_loss=30.73, generator_mel_loss=20.04, generator_kl_loss=1.965, generator_dur_loss=1.48, generator_adv_loss=2.418, generator_feat_match_loss=4.823, over 1275.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:05:53,940 INFO [train.py:845] (2/4) Start epoch 657
2024-02-24 00:08:44,794 INFO [train.py:471] (2/4) Epoch 657, batch 28, global_batch_idx: 24300, batch size: 79, loss[discriminator_loss=2.496, discriminator_real_loss=1.195, discriminator_fake_loss=1.301, generator_loss=30.42, generator_mel_loss=20.01, generator_kl_loss=1.925, generator_dur_loss=1.471, generator_adv_loss=2.496, generator_feat_match_loss=4.516, over 79.00 samples.], tot_loss[discriminator_loss=2.482, discriminator_real_loss=1.254, discriminator_fake_loss=1.228, generator_loss=30.47, generator_mel_loss=19.91, generator_kl_loss=1.915, generator_dur_loss=1.475, generator_adv_loss=2.375, generator_feat_match_loss=4.798, over 2066.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:09:29,826 INFO [train.py:845] (2/4) Start epoch 658
2024-02-24 00:12:54,528 INFO [train.py:845] (2/4) Start epoch 659
2024-02-24 00:13:37,273 INFO [train.py:471] (2/4) Epoch 659, batch 4, global_batch_idx: 24350, batch size: 56, loss[discriminator_loss=2.434, discriminator_real_loss=1.387, discriminator_fake_loss=1.046, generator_loss=30.45, generator_mel_loss=19.64, generator_kl_loss=1.862, generator_dur_loss=1.505, generator_adv_loss=2.496, generator_feat_match_loss=4.945, over 56.00 samples.], tot_loss[discriminator_loss=2.494, discriminator_real_loss=1.255, discriminator_fake_loss=1.238, generator_loss=30.82, generator_mel_loss=20.12, generator_kl_loss=1.917, generator_dur_loss=1.476, generator_adv_loss=2.455, generator_feat_match_loss=4.855, over 417.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:16:24,339 INFO [train.py:845] (2/4) Start epoch 660
2024-02-24 00:18:11,439 INFO [train.py:471] (2/4) Epoch 660, batch 17, global_batch_idx: 24400, batch size: 79, loss[discriminator_loss=2.246, discriminator_real_loss=1.132, discriminator_fake_loss=1.114, generator_loss=30.96, generator_mel_loss=19.39, generator_kl_loss=1.886, generator_dur_loss=1.488, generator_adv_loss=2.725, generator_feat_match_loss=5.469, over 79.00 samples.], tot_loss[discriminator_loss=2.377, discriminator_real_loss=1.216, discriminator_fake_loss=1.16, generator_loss=30.75, generator_mel_loss=19.65, generator_kl_loss=1.906, generator_dur_loss=1.475, generator_adv_loss=2.52, generator_feat_match_loss=5.198, over 1278.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 16.0
2024-02-24 00:18:11,440 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 00:18:19,769 INFO [train.py:534] (2/4) Epoch 660, validation: discriminator_loss=2.242, discriminator_real_loss=1.085, discriminator_fake_loss=1.157, generator_loss=32.49, generator_mel_loss=20.58, generator_kl_loss=2.052, generator_dur_loss=1.474, generator_adv_loss=2.542, generator_feat_match_loss=5.837, over 100.00 samples.
2024-02-24 00:18:19,770 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 00:20:05,017 INFO [train.py:845] (2/4) Start epoch 661
2024-02-24 00:22:58,906 INFO [train.py:471] (2/4) Epoch 661, batch 30, global_batch_idx: 24450, batch size: 65, loss[discriminator_loss=2.367, discriminator_real_loss=1.123, discriminator_fake_loss=1.245, generator_loss=30.21, generator_mel_loss=19.62, generator_kl_loss=1.847, generator_dur_loss=1.491, generator_adv_loss=2.301, generator_feat_match_loss=4.945, over 65.00 samples.], tot_loss[discriminator_loss=2.449, discriminator_real_loss=1.23, discriminator_fake_loss=1.219, generator_loss=30.48, generator_mel_loss=19.89, generator_kl_loss=1.919, generator_dur_loss=1.478, generator_adv_loss=2.388, generator_feat_match_loss=4.801, over 2083.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:23:33,398 INFO [train.py:845] (2/4) Start epoch 662
2024-02-24 00:27:03,475 INFO [train.py:845] (2/4) Start epoch 663
2024-02-24 00:27:48,107 INFO [train.py:471] (2/4) Epoch 663, batch 6, global_batch_idx: 24500, batch size: 65, loss[discriminator_loss=2.502, discriminator_real_loss=1.313, discriminator_fake_loss=1.188, generator_loss=30.23, generator_mel_loss=19.93, generator_kl_loss=1.97, generator_dur_loss=1.493, generator_adv_loss=2.238, generator_feat_match_loss=4.602, over 65.00 samples.], tot_loss[discriminator_loss=2.441, discriminator_real_loss=1.232, discriminator_fake_loss=1.209, generator_loss=30.48, generator_mel_loss=19.89, generator_kl_loss=1.921, generator_dur_loss=1.473, generator_adv_loss=2.306, generator_feat_match_loss=4.885, over 506.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:30:30,309 INFO [train.py:845] (2/4) Start epoch 664
2024-02-24 00:32:27,786 INFO [train.py:471] (2/4) Epoch 664, batch 19, global_batch_idx: 24550, batch size: 71, loss[discriminator_loss=2.488, discriminator_real_loss=1.271, discriminator_fake_loss=1.217, generator_loss=30.51, generator_mel_loss=19.81, generator_kl_loss=1.987, generator_dur_loss=1.484, generator_adv_loss=2.508, generator_feat_match_loss=4.727, over 71.00 samples.], tot_loss[discriminator_loss=2.426, discriminator_real_loss=1.2, discriminator_fake_loss=1.225, generator_loss=30.35, generator_mel_loss=19.6, generator_kl_loss=1.956, generator_dur_loss=1.476, generator_adv_loss=2.403, generator_feat_match_loss=4.912, over 1402.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:34:00,311 INFO [train.py:845] (2/4) Start epoch 665
2024-02-24 00:37:00,918 INFO [train.py:471] (2/4) Epoch 665, batch 32, global_batch_idx: 24600, batch size: 61, loss[discriminator_loss=2.285, discriminator_real_loss=1.197, discriminator_fake_loss=1.088, generator_loss=30.84, generator_mel_loss=19.46, generator_kl_loss=1.934, generator_dur_loss=1.463, generator_adv_loss=2.605, generator_feat_match_loss=5.379, over 61.00 samples.], tot_loss[discriminator_loss=2.432, discriminator_real_loss=1.217, discriminator_fake_loss=1.215, generator_loss=30.63, generator_mel_loss=19.77, generator_kl_loss=1.945, generator_dur_loss=1.476, generator_adv_loss=2.42, generator_feat_match_loss=5.013, over 2305.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:37:00,919 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 00:37:10,033 INFO [train.py:534] (2/4) Epoch 665, validation: discriminator_loss=2.248, discriminator_real_loss=1.099, discriminator_fake_loss=1.148, generator_loss=31.86, generator_mel_loss=20.2, generator_kl_loss=1.947, generator_dur_loss=1.474, generator_adv_loss=2.509, generator_feat_match_loss=5.722, over 100.00 samples.
2024-02-24 00:37:10,034 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 00:37:33,024 INFO [train.py:845] (2/4) Start epoch 666
2024-02-24 00:41:05,928 INFO [train.py:845] (2/4) Start epoch 667
2024-02-24 00:42:03,404 INFO [train.py:471] (2/4) Epoch 667, batch 8, global_batch_idx: 24650, batch size: 59, loss[discriminator_loss=2.289, discriminator_real_loss=1.098, discriminator_fake_loss=1.19, generator_loss=31.52, generator_mel_loss=19.94, generator_kl_loss=1.91, generator_dur_loss=1.497, generator_adv_loss=2.551, generator_feat_match_loss=5.625, over 59.00 samples.], tot_loss[discriminator_loss=2.465, discriminator_real_loss=1.196, discriminator_fake_loss=1.269, generator_loss=30.85, generator_mel_loss=19.89, generator_kl_loss=1.941, generator_dur_loss=1.47, generator_adv_loss=2.391, generator_feat_match_loss=5.156, over 791.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:44:32,881 INFO [train.py:845] (2/4) Start epoch 668
2024-02-24 00:46:36,381 INFO [train.py:471] (2/4) Epoch 668, batch 21, global_batch_idx: 24700, batch size: 49, loss[discriminator_loss=2.463, discriminator_real_loss=1.146, discriminator_fake_loss=1.317, generator_loss=29.94, generator_mel_loss=19.75, generator_kl_loss=1.906, generator_dur_loss=1.503, generator_adv_loss=2.191, generator_feat_match_loss=4.586, over 49.00 samples.], tot_loss[discriminator_loss=2.413, discriminator_real_loss=1.216, discriminator_fake_loss=1.197, generator_loss=30.55, generator_mel_loss=19.63, generator_kl_loss=1.9, generator_dur_loss=1.477, generator_adv_loss=2.468, generator_feat_match_loss=5.072, over 1685.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:48:05,642 INFO [train.py:845] (2/4) Start epoch 669
2024-02-24 00:51:19,280 INFO [train.py:471] (2/4) Epoch 669, batch 34, global_batch_idx: 24750, batch size: 60, loss[discriminator_loss=2.684, discriminator_real_loss=1.21, discriminator_fake_loss=1.473, generator_loss=31.33, generator_mel_loss=20.2, generator_kl_loss=1.968, generator_dur_loss=1.478, generator_adv_loss=2.682, generator_feat_match_loss=5.004, over 60.00 samples.], tot_loss[discriminator_loss=2.659, discriminator_real_loss=1.359, discriminator_fake_loss=1.3, generator_loss=30.44, generator_mel_loss=20.14, generator_kl_loss=1.936, generator_dur_loss=1.471, generator_adv_loss=2.351, generator_feat_match_loss=4.544, over 2764.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 00:51:32,137 INFO [train.py:845] (2/4) Start epoch 670
2024-02-24 00:54:58,828 INFO [train.py:845] (2/4) Start epoch 671
2024-02-24 00:56:06,435 INFO [train.py:471] (2/4) Epoch 671, batch 10, global_batch_idx: 24800, batch size: 50, loss[discriminator_loss=2.785, discriminator_real_loss=1.419, discriminator_fake_loss=1.365, generator_loss=28.53, generator_mel_loss=19.73, generator_kl_loss=1.907, generator_dur_loss=1.49, generator_adv_loss=1.86, generator_feat_match_loss=3.545, over 50.00 samples.], tot_loss[discriminator_loss=2.754, discriminator_real_loss=1.428, discriminator_fake_loss=1.327, generator_loss=28.17, generator_mel_loss=19.44, generator_kl_loss=1.945, generator_dur_loss=1.477, generator_adv_loss=1.904, generator_feat_match_loss=3.399, over 788.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 16.0
2024-02-24 00:56:06,437 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 00:56:14,206 INFO [train.py:534] (2/4) Epoch 671, validation: discriminator_loss=2.773, discriminator_real_loss=1.395, discriminator_fake_loss=1.378, generator_loss=29.3, generator_mel_loss=20.4, generator_kl_loss=2.083, generator_dur_loss=1.47, generator_adv_loss=1.803, generator_feat_match_loss=3.54, over 100.00 samples.
2024-02-24 00:56:14,207 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 00:58:35,610 INFO [train.py:845] (2/4) Start epoch 672
2024-02-24 01:00:52,631 INFO [train.py:471] (2/4) Epoch 672, batch 23, global_batch_idx: 24850, batch size: 60, loss[discriminator_loss=2.611, discriminator_real_loss=1.253, discriminator_fake_loss=1.358, generator_loss=29.71, generator_mel_loss=19.82, generator_kl_loss=1.949, generator_dur_loss=1.491, generator_adv_loss=2.389, generator_feat_match_loss=4.059, over 60.00 samples.], tot_loss[discriminator_loss=2.675, discriminator_real_loss=1.375, discriminator_fake_loss=1.3, generator_loss=29.25, generator_mel_loss=19.77, generator_kl_loss=1.947, generator_dur_loss=1.476, generator_adv_loss=2.065, generator_feat_match_loss=3.99, over 1783.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:02:02,391 INFO [train.py:845] (2/4) Start epoch 673
2024-02-24 01:05:32,296 INFO [train.py:471] (2/4) Epoch 673, batch 36, global_batch_idx: 24900, batch size: 65, loss[discriminator_loss=2.662, discriminator_real_loss=1.332, discriminator_fake_loss=1.33, generator_loss=28.93, generator_mel_loss=19.64, generator_kl_loss=1.883, generator_dur_loss=1.467, generator_adv_loss=2.121, generator_feat_match_loss=3.826, over 65.00 samples.], tot_loss[discriminator_loss=2.698, discriminator_real_loss=1.365, discriminator_fake_loss=1.333, generator_loss=29.08, generator_mel_loss=19.82, generator_kl_loss=1.907, generator_dur_loss=1.475, generator_adv_loss=2.005, generator_feat_match_loss=3.875, over 2683.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:05:32,714 INFO [train.py:845] (2/4) Start epoch 674
2024-02-24 01:08:58,027 INFO [train.py:845] (2/4) Start epoch 675
2024-02-24 01:10:12,797 INFO [train.py:471] (2/4) Epoch 675, batch 12, global_batch_idx: 24950, batch size: 67, loss[discriminator_loss=2.617, discriminator_real_loss=1.366, discriminator_fake_loss=1.252, generator_loss=30.02, generator_mel_loss=19.87, generator_kl_loss=1.964, generator_dur_loss=1.475, generator_adv_loss=2.121, generator_feat_match_loss=4.586, over 67.00 samples.], tot_loss[discriminator_loss=2.781, discriminator_real_loss=1.437, discriminator_fake_loss=1.343, generator_loss=29.86, generator_mel_loss=20.05, generator_kl_loss=1.948, generator_dur_loss=1.481, generator_adv_loss=2.242, generator_feat_match_loss=4.132, over 827.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:12:33,839 INFO [train.py:845] (2/4) Start epoch 676
2024-02-24 01:14:59,679 INFO [train.py:471] (2/4) Epoch 676, batch 25, global_batch_idx: 25000, batch size: 73, loss[discriminator_loss=2.633, discriminator_real_loss=1.309, discriminator_fake_loss=1.323, generator_loss=29.79, generator_mel_loss=20.01, generator_kl_loss=1.92, generator_dur_loss=1.456, generator_adv_loss=2.219, generator_feat_match_loss=4.191, over 73.00 samples.], tot_loss[discriminator_loss=2.609, discriminator_real_loss=1.316, discriminator_fake_loss=1.292, generator_loss=29.7, generator_mel_loss=20.09, generator_kl_loss=1.95, generator_dur_loss=1.472, generator_adv_loss=2.083, generator_feat_match_loss=4.101, over 1940.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:14:59,681 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 01:15:07,370 INFO [train.py:534] (2/4) Epoch 676, validation: discriminator_loss=2.597, discriminator_real_loss=1.37, discriminator_fake_loss=1.227, generator_loss=30.54, generator_mel_loss=20.51, generator_kl_loss=2.108, generator_dur_loss=1.471, generator_adv_loss=2.208, generator_feat_match_loss=4.246, over 100.00 samples.
2024-02-24 01:15:07,371 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 01:16:10,130 INFO [train.py:845] (2/4) Start epoch 677
2024-02-24 01:19:37,741 INFO [train.py:845] (2/4) Start epoch 678
2024-02-24 01:19:56,700 INFO [train.py:471] (2/4) Epoch 678, batch 1, global_batch_idx: 25050, batch size: 64, loss[discriminator_loss=2.715, discriminator_real_loss=1.461, discriminator_fake_loss=1.254, generator_loss=29.75, generator_mel_loss=19.94, generator_kl_loss=1.907, generator_dur_loss=1.47, generator_adv_loss=2.295, generator_feat_match_loss=4.141, over 64.00 samples.], tot_loss[discriminator_loss=2.764, discriminator_real_loss=1.48, discriminator_fake_loss=1.284, generator_loss=29.73, generator_mel_loss=19.97, generator_kl_loss=1.908, generator_dur_loss=1.479, generator_adv_loss=2.327, generator_feat_match_loss=4.041, over 122.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:23:00,040 INFO [train.py:845] (2/4) Start epoch 679
2024-02-24 01:24:36,089 INFO [train.py:471] (2/4) Epoch 679, batch 14, global_batch_idx: 25100, batch size: 95, loss[discriminator_loss=2.604, discriminator_real_loss=1.252, discriminator_fake_loss=1.352, generator_loss=30.55, generator_mel_loss=20.73, generator_kl_loss=1.995, generator_dur_loss=1.477, generator_adv_loss=2.115, generator_feat_match_loss=4.227, over 95.00 samples.], tot_loss[discriminator_loss=2.65, discriminator_real_loss=1.335, discriminator_fake_loss=1.315, generator_loss=29.83, generator_mel_loss=20.25, generator_kl_loss=1.972, generator_dur_loss=1.471, generator_adv_loss=2.043, generator_feat_match_loss=4.095, over 1248.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:26:32,475 INFO [train.py:845] (2/4) Start epoch 680
2024-02-24 01:29:06,946 INFO [train.py:471] (2/4) Epoch 680, batch 27, global_batch_idx: 25150, batch size: 60, loss[discriminator_loss=2.598, discriminator_real_loss=1.184, discriminator_fake_loss=1.415, generator_loss=29.87, generator_mel_loss=20.2, generator_kl_loss=1.922, generator_dur_loss=1.464, generator_adv_loss=2.146, generator_feat_match_loss=4.141, over 60.00 samples.], tot_loss[discriminator_loss=2.646, discriminator_real_loss=1.345, discriminator_fake_loss=1.302, generator_loss=29.79, generator_mel_loss=20.14, generator_kl_loss=1.935, generator_dur_loss=1.478, generator_adv_loss=2.127, generator_feat_match_loss=4.107, over 1903.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:29:53,321 INFO [train.py:845] (2/4) Start epoch 681
2024-02-24 01:33:21,658 INFO [train.py:845] (2/4) Start epoch 682
2024-02-24 01:33:52,785 INFO [train.py:471] (2/4) Epoch 682, batch 3, global_batch_idx: 25200, batch size: 90, loss[discriminator_loss=2.855, discriminator_real_loss=1.357, discriminator_fake_loss=1.498, generator_loss=29.23, generator_mel_loss=20.38, generator_kl_loss=1.897, generator_dur_loss=1.494, generator_adv_loss=1.936, generator_feat_match_loss=3.529, over 90.00 samples.], tot_loss[discriminator_loss=2.866, discriminator_real_loss=1.416, discriminator_fake_loss=1.451, generator_loss=29.68, generator_mel_loss=20.33, generator_kl_loss=1.906, generator_dur_loss=1.486, generator_adv_loss=2.009, generator_feat_match_loss=3.957, over 289.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:33:52,787 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 01:34:01,952 INFO [train.py:534] (2/4) Epoch 682, validation: discriminator_loss=2.722, discriminator_real_loss=1.403, discriminator_fake_loss=1.319, generator_loss=31.15, generator_mel_loss=21.48, generator_kl_loss=2.031, generator_dur_loss=1.478, generator_adv_loss=2.106, generator_feat_match_loss=4.049, over 100.00 samples.
2024-02-24 01:34:01,953 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 01:36:53,938 INFO [train.py:845] (2/4) Start epoch 683
2024-02-24 01:38:37,247 INFO [train.py:471] (2/4) Epoch 683, batch 16, global_batch_idx: 25250, batch size: 65, loss[discriminator_loss=2.545, discriminator_real_loss=1.369, discriminator_fake_loss=1.176, generator_loss=30.71, generator_mel_loss=20.41, generator_kl_loss=1.932, generator_dur_loss=1.468, generator_adv_loss=2.32, generator_feat_match_loss=4.578, over 65.00 samples.], tot_loss[discriminator_loss=2.607, discriminator_real_loss=1.319, discriminator_fake_loss=1.289, generator_loss=29.99, generator_mel_loss=20.11, generator_kl_loss=1.92, generator_dur_loss=1.475, generator_adv_loss=2.194, generator_feat_match_loss=4.291, over 1180.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 4.0
2024-02-24 01:40:22,755 INFO [train.py:845] (2/4) Start epoch 684
2024-02-24 01:43:13,587 INFO [train.py:471] (2/4) Epoch 684, batch 29, global_batch_idx: 25300, batch size: 71, loss[discriminator_loss=2.621, discriminator_real_loss=1.302, discriminator_fake_loss=1.318, generator_loss=29.8, generator_mel_loss=20.16, generator_kl_loss=1.956, generator_dur_loss=1.476, generator_adv_loss=2.057, generator_feat_match_loss=4.152, over 71.00 samples.], tot_loss[discriminator_loss=2.622, discriminator_real_loss=1.325, discriminator_fake_loss=1.297, generator_loss=29.96, generator_mel_loss=20.12, generator_kl_loss=1.917, generator_dur_loss=1.477, generator_adv_loss=2.184, generator_feat_match_loss=4.26, over 2278.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:43:51,326 INFO [train.py:845] (2/4) Start epoch 685
2024-02-24 01:47:23,351 INFO [train.py:845] (2/4) Start epoch 686
2024-02-24 01:48:11,018 INFO [train.py:471] (2/4) Epoch 686, batch 5, global_batch_idx: 25350, batch size: 110, loss[discriminator_loss=2.477, discriminator_real_loss=1.264, discriminator_fake_loss=1.213, generator_loss=30.91, generator_mel_loss=20.15, generator_kl_loss=1.951, generator_dur_loss=1.455, generator_adv_loss=2.436, generator_feat_match_loss=4.918, over 110.00 samples.], tot_loss[discriminator_loss=2.556, discriminator_real_loss=1.295, discriminator_fake_loss=1.261, generator_loss=30.8, generator_mel_loss=20.23, generator_kl_loss=1.954, generator_dur_loss=1.475, generator_adv_loss=2.424, generator_feat_match_loss=4.713, over 503.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 4.0
2024-02-24 01:50:52,872 INFO [train.py:845] (2/4) Start epoch 687
2024-02-24 01:52:44,409 INFO [train.py:471] (2/4) Epoch 687, batch 18, global_batch_idx: 25400, batch size: 51, loss[discriminator_loss=2.656, discriminator_real_loss=1.379, discriminator_fake_loss=1.278, generator_loss=30.2, generator_mel_loss=20.53, generator_kl_loss=1.892, generator_dur_loss=1.466, generator_adv_loss=2.074, generator_feat_match_loss=4.242, over 51.00 samples.], tot_loss[discriminator_loss=2.595, discriminator_real_loss=1.308, discriminator_fake_loss=1.287, generator_loss=29.79, generator_mel_loss=20.15, generator_kl_loss=1.922, generator_dur_loss=1.477, generator_adv_loss=2.089, generator_feat_match_loss=4.147, over 1110.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:52:44,411 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 01:52:52,986 INFO [train.py:534] (2/4) Epoch 687, validation: discriminator_loss=2.567, discriminator_real_loss=1.235, discriminator_fake_loss=1.332, generator_loss=30.84, generator_mel_loss=20.99, generator_kl_loss=2.061, generator_dur_loss=1.475, generator_adv_loss=1.95, generator_feat_match_loss=4.365, over 100.00 samples.
2024-02-24 01:52:52,987 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 01:54:32,401 INFO [train.py:845] (2/4) Start epoch 688
2024-02-24 01:57:32,908 INFO [train.py:471] (2/4) Epoch 688, batch 31, global_batch_idx: 25450, batch size: 51, loss[discriminator_loss=2.727, discriminator_real_loss=1.367, discriminator_fake_loss=1.359, generator_loss=29.68, generator_mel_loss=20.19, generator_kl_loss=1.988, generator_dur_loss=1.507, generator_adv_loss=2.105, generator_feat_match_loss=3.883, over 51.00 samples.], tot_loss[discriminator_loss=2.675, discriminator_real_loss=1.352, discriminator_fake_loss=1.323, generator_loss=29.94, generator_mel_loss=20.36, generator_kl_loss=1.918, generator_dur_loss=1.472, generator_adv_loss=2.104, generator_feat_match_loss=4.084, over 2376.00 samples.], cur_lr_g: 1.84e-04, cur_lr_d: 1.84e-04, grad_scale: 8.0
2024-02-24 01:58:02,465 INFO [train.py:845] (2/4) Start epoch 689
2024-02-24 02:01:27,728 INFO [train.py:845] (2/4) Start epoch 690
2024-02-24 02:02:18,872 INFO [train.py:471] (2/4) Epoch 690, batch 7, global_batch_idx: 25500, batch size: 69, loss[discriminator_loss=2.551, discriminator_real_loss=1.4, discriminator_fake_loss=1.15, generator_loss=30.64, generator_mel_loss=20.25, generator_kl_loss=1.788, generator_dur_loss=1.473, generator_adv_loss=2.443, generator_feat_match_loss=4.684, over 69.00 samples.], tot_loss[discriminator_loss=2.613, discriminator_real_loss=1.3, discriminator_fake_loss=1.313, generator_loss=30.34, generator_mel_loss=20.24, generator_kl_loss=1.918, generator_dur_loss=1.477, generator_adv_loss=2.279, generator_feat_match_loss=4.431, over 515.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:05:00,048 INFO [train.py:845] (2/4) Start epoch 691
2024-02-24 02:07:01,624 INFO [train.py:471] (2/4) Epoch 691, batch 20, global_batch_idx: 25550, batch size: 58, loss[discriminator_loss=2.445, discriminator_real_loss=1.296, discriminator_fake_loss=1.149, generator_loss=31, generator_mel_loss=19.93, generator_kl_loss=1.959, generator_dur_loss=1.469, generator_adv_loss=2.453, generator_feat_match_loss=5.188, over 58.00 samples.], tot_loss[discriminator_loss=2.586, discriminator_real_loss=1.305, discriminator_fake_loss=1.281, generator_loss=30.61, generator_mel_loss=20.3, generator_kl_loss=1.951, generator_dur_loss=1.476, generator_adv_loss=2.313, generator_feat_match_loss=4.562, over 1534.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 02:08:31,196 INFO [train.py:845] (2/4) Start epoch 692
2024-02-24 02:11:45,047 INFO [train.py:471] (2/4) Epoch 692, batch 33, global_batch_idx: 25600, batch size: 63, loss[discriminator_loss=2.535, discriminator_real_loss=1.286, discriminator_fake_loss=1.249, generator_loss=29.91, generator_mel_loss=19.97, generator_kl_loss=1.982, generator_dur_loss=1.454, generator_adv_loss=2.145, generator_feat_match_loss=4.352, over 63.00 samples.], tot_loss[discriminator_loss=2.587, discriminator_real_loss=1.31, discriminator_fake_loss=1.277, generator_loss=30.04, generator_mel_loss=20.17, generator_kl_loss=1.931, generator_dur_loss=1.47, generator_adv_loss=2.153, generator_feat_match_loss=4.31, over 2668.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:11:45,049 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 02:11:54,226 INFO [train.py:534] (2/4) Epoch 692, validation: discriminator_loss=2.496, discriminator_real_loss=1.111, discriminator_fake_loss=1.385, generator_loss=30.57, generator_mel_loss=20.74, generator_kl_loss=2.04, generator_dur_loss=1.475, generator_adv_loss=1.97, generator_feat_match_loss=4.353, over 100.00 samples.
2024-02-24 02:11:54,227 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 02:12:06,586 INFO [train.py:845] (2/4) Start epoch 693
2024-02-24 02:15:32,838 INFO [train.py:845] (2/4) Start epoch 694
2024-02-24 02:16:33,514 INFO [train.py:471] (2/4) Epoch 694, batch 9, global_batch_idx: 25650, batch size: 58, loss[discriminator_loss=2.742, discriminator_real_loss=1.368, discriminator_fake_loss=1.374, generator_loss=29.06, generator_mel_loss=19.49, generator_kl_loss=1.915, generator_dur_loss=1.476, generator_adv_loss=2.131, generator_feat_match_loss=4.047, over 58.00 samples.], tot_loss[discriminator_loss=2.501, discriminator_real_loss=1.291, discriminator_fake_loss=1.21, generator_loss=30.42, generator_mel_loss=20.07, generator_kl_loss=1.92, generator_dur_loss=1.471, generator_adv_loss=2.245, generator_feat_match_loss=4.718, over 775.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 02:19:00,852 INFO [train.py:845] (2/4) Start epoch 695
2024-02-24 02:21:13,853 INFO [train.py:471] (2/4) Epoch 695, batch 22, global_batch_idx: 25700, batch size: 95, loss[discriminator_loss=2.604, discriminator_real_loss=1.277, discriminator_fake_loss=1.326, generator_loss=29.95, generator_mel_loss=20.26, generator_kl_loss=1.916, generator_dur_loss=1.462, generator_adv_loss=2.188, generator_feat_match_loss=4.125, over 95.00 samples.], tot_loss[discriminator_loss=2.601, discriminator_real_loss=1.317, discriminator_fake_loss=1.284, generator_loss=30.06, generator_mel_loss=20.29, generator_kl_loss=1.926, generator_dur_loss=1.475, generator_adv_loss=2.13, generator_feat_match_loss=4.241, over 1713.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:22:26,414 INFO [train.py:845] (2/4) Start epoch 696
2024-02-24 02:25:38,243 INFO [train.py:471] (2/4) Epoch 696, batch 35, global_batch_idx: 25750, batch size: 55, loss[discriminator_loss=2.531, discriminator_real_loss=1.185, discriminator_fake_loss=1.347, generator_loss=30.14, generator_mel_loss=20.04, generator_kl_loss=1.959, generator_dur_loss=1.528, generator_adv_loss=2.391, generator_feat_match_loss=4.227, over 55.00 samples.], tot_loss[discriminator_loss=2.602, discriminator_real_loss=1.306, discriminator_fake_loss=1.296, generator_loss=30.31, generator_mel_loss=20.22, generator_kl_loss=1.916, generator_dur_loss=1.477, generator_adv_loss=2.257, generator_feat_match_loss=4.44, over 2393.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 02:25:46,358 INFO [train.py:845] (2/4) Start epoch 697
2024-02-24 02:29:16,785 INFO [train.py:845] (2/4) Start epoch 698
2024-02-24 02:30:36,828 INFO [train.py:471] (2/4) Epoch 698, batch 11, global_batch_idx: 25800, batch size: 126, loss[discriminator_loss=2.609, discriminator_real_loss=1.3, discriminator_fake_loss=1.309, generator_loss=29.87, generator_mel_loss=20.29, generator_kl_loss=1.937, generator_dur_loss=1.449, generator_adv_loss=2.117, generator_feat_match_loss=4.078, over 126.00 samples.], tot_loss[discriminator_loss=2.582, discriminator_real_loss=1.295, discriminator_fake_loss=1.287, generator_loss=30.26, generator_mel_loss=20.35, generator_kl_loss=1.934, generator_dur_loss=1.471, generator_adv_loss=2.137, generator_feat_match_loss=4.359, over 1167.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:30:36,830 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 02:30:45,279 INFO [train.py:534] (2/4) Epoch 698, validation: discriminator_loss=2.531, discriminator_real_loss=1.305, discriminator_fake_loss=1.226, generator_loss=30.77, generator_mel_loss=20.74, generator_kl_loss=2.004, generator_dur_loss=1.481, generator_adv_loss=2.182, generator_feat_match_loss=4.356, over 100.00 samples.
2024-02-24 02:30:45,280 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 02:32:56,630 INFO [train.py:845] (2/4) Start epoch 699
2024-02-24 02:35:12,084 INFO [train.py:471] (2/4) Epoch 699, batch 24, global_batch_idx: 25850, batch size: 126, loss[discriminator_loss=2.609, discriminator_real_loss=1.429, discriminator_fake_loss=1.182, generator_loss=31.18, generator_mel_loss=20.42, generator_kl_loss=1.912, generator_dur_loss=1.471, generator_adv_loss=2.373, generator_feat_match_loss=5.008, over 126.00 samples.], tot_loss[discriminator_loss=2.602, discriminator_real_loss=1.341, discriminator_fake_loss=1.261, generator_loss=30.39, generator_mel_loss=20.13, generator_kl_loss=1.909, generator_dur_loss=1.477, generator_adv_loss=2.305, generator_feat_match_loss=4.569, over 1793.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 02:36:22,746 INFO [train.py:845] (2/4) Start epoch 700
2024-02-24 02:39:49,222 INFO [train.py:845] (2/4) Start epoch 701
2024-02-24 02:40:02,138 INFO [train.py:471] (2/4) Epoch 701, batch 0, global_batch_idx: 25900, batch size: 85, loss[discriminator_loss=2.648, discriminator_real_loss=1.438, discriminator_fake_loss=1.212, generator_loss=30.05, generator_mel_loss=20.45, generator_kl_loss=1.938, generator_dur_loss=1.472, generator_adv_loss=2.021, generator_feat_match_loss=4.172, over 85.00 samples.], tot_loss[discriminator_loss=2.648, discriminator_real_loss=1.438, discriminator_fake_loss=1.212, generator_loss=30.05, generator_mel_loss=20.45, generator_kl_loss=1.938, generator_dur_loss=1.472, generator_adv_loss=2.021, generator_feat_match_loss=4.172, over 85.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:43:18,350 INFO [train.py:845] (2/4) Start epoch 702
2024-02-24 02:44:43,440 INFO [train.py:471] (2/4) Epoch 702, batch 13, global_batch_idx: 25950, batch size: 55, loss[discriminator_loss=2.646, discriminator_real_loss=1.188, discriminator_fake_loss=1.459, generator_loss=30.15, generator_mel_loss=20.14, generator_kl_loss=1.899, generator_dur_loss=1.521, generator_adv_loss=2.318, generator_feat_match_loss=4.273, over 55.00 samples.], tot_loss[discriminator_loss=2.656, discriminator_real_loss=1.332, discriminator_fake_loss=1.324, generator_loss=30.25, generator_mel_loss=20.37, generator_kl_loss=1.926, generator_dur_loss=1.483, generator_adv_loss=2.177, generator_feat_match_loss=4.29, over 886.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:46:48,089 INFO [train.py:845] (2/4) Start epoch 703
2024-02-24 02:49:24,984 INFO [train.py:471] (2/4) Epoch 703, batch 26, global_batch_idx: 26000, batch size: 63, loss[discriminator_loss=2.617, discriminator_real_loss=1.239, discriminator_fake_loss=1.377, generator_loss=29.39, generator_mel_loss=19.85, generator_kl_loss=1.927, generator_dur_loss=1.493, generator_adv_loss=2.049, generator_feat_match_loss=4.07, over 63.00 samples.], tot_loss[discriminator_loss=2.576, discriminator_real_loss=1.284, discriminator_fake_loss=1.292, generator_loss=30.32, generator_mel_loss=20.18, generator_kl_loss=1.938, generator_dur_loss=1.48, generator_adv_loss=2.249, generator_feat_match_loss=4.474, over 1846.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:49:24,986 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 02:49:33,686 INFO [train.py:534] (2/4) Epoch 703, validation: discriminator_loss=2.517, discriminator_real_loss=1.243, discriminator_fake_loss=1.275, generator_loss=31.21, generator_mel_loss=20.55, generator_kl_loss=2.055, generator_dur_loss=1.477, generator_adv_loss=2.355, generator_feat_match_loss=4.774, over 100.00 samples.
2024-02-24 02:49:33,687 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 02:50:34,648 INFO [train.py:845] (2/4) Start epoch 704
2024-02-24 02:54:03,521 INFO [train.py:845] (2/4) Start epoch 705
2024-02-24 02:54:25,658 INFO [train.py:471] (2/4) Epoch 705, batch 2, global_batch_idx: 26050, batch size: 50, loss[discriminator_loss=2.422, discriminator_real_loss=1.088, discriminator_fake_loss=1.334, generator_loss=31.49, generator_mel_loss=20.16, generator_kl_loss=2.006, generator_dur_loss=1.503, generator_adv_loss=2.471, generator_feat_match_loss=5.344, over 50.00 samples.], tot_loss[discriminator_loss=2.422, discriminator_real_loss=1.223, discriminator_fake_loss=1.2, generator_loss=30.97, generator_mel_loss=20, generator_kl_loss=1.977, generator_dur_loss=1.485, generator_adv_loss=2.372, generator_feat_match_loss=5.132, over 197.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 02:57:32,053 INFO [train.py:845] (2/4) Start epoch 706
2024-02-24 02:59:06,165 INFO [train.py:471] (2/4) Epoch 706, batch 15, global_batch_idx: 26100, batch size: 59, loss[discriminator_loss=2.684, discriminator_real_loss=1.349, discriminator_fake_loss=1.334, generator_loss=30.54, generator_mel_loss=20.71, generator_kl_loss=1.981, generator_dur_loss=1.499, generator_adv_loss=2.158, generator_feat_match_loss=4.195, over 59.00 samples.], tot_loss[discriminator_loss=2.58, discriminator_real_loss=1.312, discriminator_fake_loss=1.268, generator_loss=30.23, generator_mel_loss=20.35, generator_kl_loss=1.952, generator_dur_loss=1.474, generator_adv_loss=2.111, generator_feat_match_loss=4.347, over 1237.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:00:58,121 INFO [train.py:845] (2/4) Start epoch 707
2024-02-24 03:03:40,066 INFO [train.py:471] (2/4) Epoch 707, batch 28, global_batch_idx: 26150, batch size: 67, loss[discriminator_loss=2.617, discriminator_real_loss=1.402, discriminator_fake_loss=1.214, generator_loss=30.23, generator_mel_loss=20.5, generator_kl_loss=1.988, generator_dur_loss=1.498, generator_adv_loss=2.004, generator_feat_match_loss=4.238, over 67.00 samples.], tot_loss[discriminator_loss=2.639, discriminator_real_loss=1.336, discriminator_fake_loss=1.303, generator_loss=30.25, generator_mel_loss=20.43, generator_kl_loss=1.932, generator_dur_loss=1.478, generator_adv_loss=2.138, generator_feat_match_loss=4.278, over 1887.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:04:26,827 INFO [train.py:845] (2/4) Start epoch 708
2024-02-24 03:07:59,616 INFO [train.py:845] (2/4) Start epoch 709
2024-02-24 03:08:32,342 INFO [train.py:471] (2/4) Epoch 709, batch 4, global_batch_idx: 26200, batch size: 58, loss[discriminator_loss=2.695, discriminator_real_loss=1.256, discriminator_fake_loss=1.44, generator_loss=29.24, generator_mel_loss=19.69, generator_kl_loss=1.784, generator_dur_loss=1.492, generator_adv_loss=2.092, generator_feat_match_loss=4.176, over 58.00 samples.], tot_loss[discriminator_loss=2.625, discriminator_real_loss=1.33, discriminator_fake_loss=1.295, generator_loss=29.89, generator_mel_loss=20.01, generator_kl_loss=1.867, generator_dur_loss=1.488, generator_adv_loss=2.197, generator_feat_match_loss=4.327, over 305.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:08:32,343 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 03:08:41,299 INFO [train.py:534] (2/4) Epoch 709, validation: discriminator_loss=2.635, discriminator_real_loss=1.287, discriminator_fake_loss=1.347, generator_loss=30.21, generator_mel_loss=20.54, generator_kl_loss=2.063, generator_dur_loss=1.469, generator_adv_loss=1.94, generator_feat_match_loss=4.192, over 100.00 samples.
2024-02-24 03:08:41,300 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 03:11:31,830 INFO [train.py:845] (2/4) Start epoch 710
2024-02-24 03:13:25,668 INFO [train.py:471] (2/4) Epoch 710, batch 17, global_batch_idx: 26250, batch size: 126, loss[discriminator_loss=2.578, discriminator_real_loss=1.164, discriminator_fake_loss=1.414, generator_loss=30.46, generator_mel_loss=20.39, generator_kl_loss=2.01, generator_dur_loss=1.451, generator_adv_loss=2.252, generator_feat_match_loss=4.355, over 126.00 samples.], tot_loss[discriminator_loss=2.57, discriminator_real_loss=1.289, discriminator_fake_loss=1.28, generator_loss=30.51, generator_mel_loss=20.28, generator_kl_loss=1.93, generator_dur_loss=1.466, generator_adv_loss=2.263, generator_feat_match_loss=4.573, over 1309.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 03:15:04,284 INFO [train.py:845] (2/4) Start epoch 711
2024-02-24 03:18:09,964 INFO [train.py:471] (2/4) Epoch 711, batch 30, global_batch_idx: 26300, batch size: 49, loss[discriminator_loss=2.566, discriminator_real_loss=1.399, discriminator_fake_loss=1.166, generator_loss=29.74, generator_mel_loss=19.69, generator_kl_loss=2.036, generator_dur_loss=1.477, generator_adv_loss=2.098, generator_feat_match_loss=4.438, over 49.00 samples.], tot_loss[discriminator_loss=2.56, discriminator_real_loss=1.286, discriminator_fake_loss=1.274, generator_loss=30.11, generator_mel_loss=20.13, generator_kl_loss=1.942, generator_dur_loss=1.477, generator_adv_loss=2.173, generator_feat_match_loss=4.383, over 2200.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:18:38,254 INFO [train.py:845] (2/4) Start epoch 712
2024-02-24 03:22:05,954 INFO [train.py:845] (2/4) Start epoch 713
2024-02-24 03:22:54,945 INFO [train.py:471] (2/4) Epoch 713, batch 6, global_batch_idx: 26350, batch size: 153, loss[discriminator_loss=2.629, discriminator_real_loss=1.217, discriminator_fake_loss=1.413, generator_loss=30.22, generator_mel_loss=20.15, generator_kl_loss=1.997, generator_dur_loss=1.457, generator_adv_loss=2.229, generator_feat_match_loss=4.395, over 153.00 samples.], tot_loss[discriminator_loss=2.655, discriminator_real_loss=1.315, discriminator_fake_loss=1.34, generator_loss=29.71, generator_mel_loss=20, generator_kl_loss=1.942, generator_dur_loss=1.476, generator_adv_loss=2.153, generator_feat_match_loss=4.134, over 565.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:25:37,863 INFO [train.py:845] (2/4) Start epoch 714
2024-02-24 03:27:35,539 INFO [train.py:471] (2/4) Epoch 714, batch 19, global_batch_idx: 26400, batch size: 85, loss[discriminator_loss=2.504, discriminator_real_loss=1.364, discriminator_fake_loss=1.139, generator_loss=30.36, generator_mel_loss=19.9, generator_kl_loss=2.056, generator_dur_loss=1.467, generator_adv_loss=2.248, generator_feat_match_loss=4.688, over 85.00 samples.], tot_loss[discriminator_loss=2.632, discriminator_real_loss=1.375, discriminator_fake_loss=1.257, generator_loss=30.31, generator_mel_loss=20.1, generator_kl_loss=1.935, generator_dur_loss=1.471, generator_adv_loss=2.294, generator_feat_match_loss=4.513, over 1487.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:27:35,540 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 03:27:43,863 INFO [train.py:534] (2/4) Epoch 714, validation: discriminator_loss=2.401, discriminator_real_loss=1.193, discriminator_fake_loss=1.208, generator_loss=31.51, generator_mel_loss=20.91, generator_kl_loss=2.072, generator_dur_loss=1.469, generator_adv_loss=2.182, generator_feat_match_loss=4.871, over 100.00 samples.
2024-02-24 03:27:43,864 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 03:29:17,444 INFO [train.py:845] (2/4) Start epoch 715
2024-02-24 03:32:29,881 INFO [train.py:471] (2/4) Epoch 715, batch 32, global_batch_idx: 26450, batch size: 67, loss[discriminator_loss=2.6, discriminator_real_loss=1.334, discriminator_fake_loss=1.266, generator_loss=30.03, generator_mel_loss=20.21, generator_kl_loss=1.877, generator_dur_loss=1.472, generator_adv_loss=2.133, generator_feat_match_loss=4.332, over 67.00 samples.], tot_loss[discriminator_loss=2.58, discriminator_real_loss=1.295, discriminator_fake_loss=1.285, generator_loss=30.12, generator_mel_loss=20.15, generator_kl_loss=1.956, generator_dur_loss=1.471, generator_adv_loss=2.149, generator_feat_match_loss=4.386, over 2447.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:32:46,904 INFO [train.py:845] (2/4) Start epoch 716
2024-02-24 03:36:12,891 INFO [train.py:845] (2/4) Start epoch 717
2024-02-24 03:37:04,814 INFO [train.py:471] (2/4) Epoch 717, batch 8, global_batch_idx: 26500, batch size: 56, loss[discriminator_loss=2.648, discriminator_real_loss=1.141, discriminator_fake_loss=1.509, generator_loss=30.41, generator_mel_loss=19.9, generator_kl_loss=2.07, generator_dur_loss=1.488, generator_adv_loss=2.217, generator_feat_match_loss=4.73, over 56.00 samples.], tot_loss[discriminator_loss=2.602, discriminator_real_loss=1.321, discriminator_fake_loss=1.281, generator_loss=30.66, generator_mel_loss=20.19, generator_kl_loss=1.937, generator_dur_loss=1.475, generator_adv_loss=2.315, generator_feat_match_loss=4.744, over 602.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:39:38,630 INFO [train.py:845] (2/4) Start epoch 718
2024-02-24 03:41:50,074 INFO [train.py:471] (2/4) Epoch 718, batch 21, global_batch_idx: 26550, batch size: 126, loss[discriminator_loss=2.623, discriminator_real_loss=1.317, discriminator_fake_loss=1.306, generator_loss=30.83, generator_mel_loss=20.74, generator_kl_loss=2.056, generator_dur_loss=1.467, generator_adv_loss=2.109, generator_feat_match_loss=4.457, over 126.00 samples.], tot_loss[discriminator_loss=2.628, discriminator_real_loss=1.329, discriminator_fake_loss=1.299, generator_loss=29.95, generator_mel_loss=20.2, generator_kl_loss=1.94, generator_dur_loss=1.466, generator_adv_loss=2.127, generator_feat_match_loss=4.215, over 1702.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:43:08,506 INFO [train.py:845] (2/4) Start epoch 719
2024-02-24 03:46:26,614 INFO [train.py:471] (2/4) Epoch 719, batch 34, global_batch_idx: 26600, batch size: 49, loss[discriminator_loss=2.646, discriminator_real_loss=1.301, discriminator_fake_loss=1.346, generator_loss=31.02, generator_mel_loss=20.43, generator_kl_loss=1.929, generator_dur_loss=1.496, generator_adv_loss=2.316, generator_feat_match_loss=4.844, over 49.00 samples.], tot_loss[discriminator_loss=2.616, discriminator_real_loss=1.333, discriminator_fake_loss=1.283, generator_loss=30.1, generator_mel_loss=20.23, generator_kl_loss=1.939, generator_dur_loss=1.473, generator_adv_loss=2.157, generator_feat_match_loss=4.302, over 2497.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:46:26,615 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 03:46:35,548 INFO [train.py:534] (2/4) Epoch 719, validation: discriminator_loss=2.669, discriminator_real_loss=1.318, discriminator_fake_loss=1.35, generator_loss=30.97, generator_mel_loss=21.19, generator_kl_loss=2.045, generator_dur_loss=1.477, generator_adv_loss=1.949, generator_feat_match_loss=4.31, over 100.00 samples.
2024-02-24 03:46:35,549 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 03:46:45,660 INFO [train.py:845] (2/4) Start epoch 720
2024-02-24 03:50:11,719 INFO [train.py:845] (2/4) Start epoch 721
2024-02-24 03:51:15,782 INFO [train.py:471] (2/4) Epoch 721, batch 10, global_batch_idx: 26650, batch size: 64, loss[discriminator_loss=2.48, discriminator_real_loss=1.312, discriminator_fake_loss=1.168, generator_loss=30.56, generator_mel_loss=19.95, generator_kl_loss=1.971, generator_dur_loss=1.471, generator_adv_loss=2.496, generator_feat_match_loss=4.676, over 64.00 samples.], tot_loss[discriminator_loss=2.542, discriminator_real_loss=1.258, discriminator_fake_loss=1.285, generator_loss=30.71, generator_mel_loss=20.18, generator_kl_loss=1.954, generator_dur_loss=1.466, generator_adv_loss=2.352, generator_feat_match_loss=4.754, over 872.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 03:53:32,811 INFO [train.py:845] (2/4) Start epoch 722
2024-02-24 03:55:53,684 INFO [train.py:471] (2/4) Epoch 722, batch 23, global_batch_idx: 26700, batch size: 64, loss[discriminator_loss=2.477, discriminator_real_loss=1.4, discriminator_fake_loss=1.076, generator_loss=31.18, generator_mel_loss=20.01, generator_kl_loss=1.872, generator_dur_loss=1.477, generator_adv_loss=2.498, generator_feat_match_loss=5.32, over 64.00 samples.], tot_loss[discriminator_loss=2.54, discriminator_real_loss=1.282, discriminator_fake_loss=1.258, generator_loss=30.63, generator_mel_loss=20.12, generator_kl_loss=1.934, generator_dur_loss=1.476, generator_adv_loss=2.355, generator_feat_match_loss=4.749, over 1834.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 03:57:01,267 INFO [train.py:845] (2/4) Start epoch 723
2024-02-24 04:00:33,891 INFO [train.py:471] (2/4) Epoch 723, batch 36, global_batch_idx: 26750, batch size: 65, loss[discriminator_loss=2.521, discriminator_real_loss=1.195, discriminator_fake_loss=1.326, generator_loss=29.47, generator_mel_loss=19.64, generator_kl_loss=1.938, generator_dur_loss=1.471, generator_adv_loss=2.072, generator_feat_match_loss=4.348, over 65.00 samples.], tot_loss[discriminator_loss=2.558, discriminator_real_loss=1.291, discriminator_fake_loss=1.267, generator_loss=30.46, generator_mel_loss=20.15, generator_kl_loss=1.945, generator_dur_loss=1.468, generator_adv_loss=2.291, generator_feat_match_loss=4.607, over 2829.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 04:00:34,339 INFO [train.py:845] (2/4) Start epoch 724
2024-02-24 04:04:03,200 INFO [train.py:845] (2/4) Start epoch 725
2024-02-24 04:05:19,273 INFO [train.py:471] (2/4) Epoch 725, batch 12, global_batch_idx: 26800, batch size: 65, loss[discriminator_loss=2.457, discriminator_real_loss=1.219, discriminator_fake_loss=1.237, generator_loss=30.67, generator_mel_loss=20.32, generator_kl_loss=1.839, generator_dur_loss=1.49, generator_adv_loss=2.297, generator_feat_match_loss=4.727, over 65.00 samples.], tot_loss[discriminator_loss=2.53, discriminator_real_loss=1.275, discriminator_fake_loss=1.256, generator_loss=30.7, generator_mel_loss=20.32, generator_kl_loss=1.925, generator_dur_loss=1.471, generator_adv_loss=2.314, generator_feat_match_loss=4.665, over 926.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 04:05:19,274 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 04:05:28,168 INFO [train.py:534] (2/4) Epoch 725, validation: discriminator_loss=2.465, discriminator_real_loss=1.132, discriminator_fake_loss=1.333, generator_loss=30.66, generator_mel_loss=20.64, generator_kl_loss=2.01, generator_dur_loss=1.475, generator_adv_loss=1.965, generator_feat_match_loss=4.567, over 100.00 samples.
2024-02-24 04:05:28,169 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 04:07:42,295 INFO [train.py:845] (2/4) Start epoch 726
2024-02-24 04:10:07,099 INFO [train.py:471] (2/4) Epoch 726, batch 25, global_batch_idx: 26850, batch size: 53, loss[discriminator_loss=2.639, discriminator_real_loss=1.447, discriminator_fake_loss=1.191, generator_loss=29.66, generator_mel_loss=19.66, generator_kl_loss=1.941, generator_dur_loss=1.502, generator_adv_loss=2.344, generator_feat_match_loss=4.207, over 53.00 samples.], tot_loss[discriminator_loss=2.542, discriminator_real_loss=1.286, discriminator_fake_loss=1.256, generator_loss=30.38, generator_mel_loss=20.05, generator_kl_loss=1.936, generator_dur_loss=1.473, generator_adv_loss=2.289, generator_feat_match_loss=4.629, over 1918.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 04:11:10,551 INFO [train.py:845] (2/4) Start epoch 727
2024-02-24 04:14:37,512 INFO [train.py:845] (2/4) Start epoch 728
2024-02-24 04:14:57,071 INFO [train.py:471] (2/4) Epoch 728, batch 1, global_batch_idx: 26900, batch size: 56, loss[discriminator_loss=2.434, discriminator_real_loss=1.256, discriminator_fake_loss=1.178, generator_loss=30.6, generator_mel_loss=20.17, generator_kl_loss=1.964, generator_dur_loss=1.47, generator_adv_loss=2.271, generator_feat_match_loss=4.727, over 56.00 samples.], tot_loss[discriminator_loss=2.419, discriminator_real_loss=1.221, discriminator_fake_loss=1.199, generator_loss=30.67, generator_mel_loss=20.02, generator_kl_loss=1.943, generator_dur_loss=1.471, generator_adv_loss=2.343, generator_feat_match_loss=4.89, over 146.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 04:18:06,161 INFO [train.py:845] (2/4) Start epoch 729
2024-02-24 04:19:37,759 INFO [train.py:471] (2/4) Epoch 729, batch 14, global_batch_idx: 26950, batch size: 58, loss[discriminator_loss=2.566, discriminator_real_loss=1.258, discriminator_fake_loss=1.31, generator_loss=29.77, generator_mel_loss=19.64, generator_kl_loss=1.982, generator_dur_loss=1.494, generator_adv_loss=2.164, generator_feat_match_loss=4.492, over 58.00 samples.], tot_loss[discriminator_loss=2.597, discriminator_real_loss=1.319, discriminator_fake_loss=1.278, generator_loss=30.29, generator_mel_loss=20, generator_kl_loss=1.94, generator_dur_loss=1.472, generator_adv_loss=2.236, generator_feat_match_loss=4.644, over 1207.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 04:21:39,654 INFO [train.py:845] (2/4) Start epoch 730
2024-02-24 04:24:22,672 INFO [train.py:471] (2/4) Epoch 730, batch 27, global_batch_idx: 27000, batch size: 55, loss[discriminator_loss=2.557, discriminator_real_loss=1.283, discriminator_fake_loss=1.273, generator_loss=29.76, generator_mel_loss=19.96, generator_kl_loss=1.824, generator_dur_loss=1.476, generator_adv_loss=2.213, generator_feat_match_loss=4.281, over 55.00 samples.], tot_loss[discriminator_loss=2.581, discriminator_real_loss=1.293, discriminator_fake_loss=1.287, generator_loss=30.61, generator_mel_loss=20.31, generator_kl_loss=1.948, generator_dur_loss=1.469, generator_adv_loss=2.275, generator_feat_match_loss=4.604, over 2091.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 04:24:22,673 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 04:24:31,406 INFO [train.py:534] (2/4) Epoch 730, validation: discriminator_loss=2.456, discriminator_real_loss=1.185, discriminator_fake_loss=1.272, generator_loss=31.51, generator_mel_loss=21.07, generator_kl_loss=2.036, generator_dur_loss=1.475, generator_adv_loss=2.24, generator_feat_match_loss=4.69, over 100.00 samples.
2024-02-24 04:24:31,406 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 04:25:19,517 INFO [train.py:845] (2/4) Start epoch 731
2024-02-24 04:28:48,510 INFO [train.py:845] (2/4) Start epoch 732
2024-02-24 04:29:20,211 INFO [train.py:471] (2/4) Epoch 732, batch 3, global_batch_idx: 27050, batch size: 69, loss[discriminator_loss=2.492, discriminator_real_loss=1.339, discriminator_fake_loss=1.154, generator_loss=29.8, generator_mel_loss=19.8, generator_kl_loss=1.97, generator_dur_loss=1.467, generator_adv_loss=2.141, generator_feat_match_loss=4.43, over 69.00 samples.], tot_loss[discriminator_loss=2.592, discriminator_real_loss=1.318, discriminator_fake_loss=1.274, generator_loss=30.01, generator_mel_loss=20.08, generator_kl_loss=1.964, generator_dur_loss=1.469, generator_adv_loss=2.14, generator_feat_match_loss=4.356, over 326.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 4.0
2024-02-24 04:32:12,242 INFO [train.py:845] (2/4) Start epoch 733
2024-02-24 04:33:50,840 INFO [train.py:471] (2/4) Epoch 733, batch 16, global_batch_idx: 27100, batch size: 50, loss[discriminator_loss=2.566, discriminator_real_loss=1.384, discriminator_fake_loss=1.184, generator_loss=30.4, generator_mel_loss=20.15, generator_kl_loss=1.91, generator_dur_loss=1.487, generator_adv_loss=2.473, generator_feat_match_loss=4.383, over 50.00 samples.], tot_loss[discriminator_loss=2.566, discriminator_real_loss=1.298, discriminator_fake_loss=1.268, generator_loss=30.31, generator_mel_loss=20.06, generator_kl_loss=1.951, generator_dur_loss=1.474, generator_adv_loss=2.31, generator_feat_match_loss=4.522, over 1165.00 samples.], cur_lr_g: 1.83e-04, cur_lr_d: 1.83e-04, grad_scale: 8.0
2024-02-24 04:35:37,693 INFO [train.py:845] (2/4) Start epoch 734
2024-02-24 04:38:20,760 INFO [train.py:471] (2/4) Epoch 734, batch 29, global_batch_idx: 27150, batch size: 76, loss[discriminator_loss=2.637, discriminator_real_loss=1.143, discriminator_fake_loss=1.494, generator_loss=30.59, generator_mel_loss=20.58, generator_kl_loss=2.077, generator_dur_loss=1.472, generator_adv_loss=2.088, generator_feat_match_loss=4.375, over 76.00 samples.], tot_loss[discriminator_loss=2.551, discriminator_real_loss=1.296, discriminator_fake_loss=1.254, generator_loss=30.25, generator_mel_loss=20.08, generator_kl_loss=1.933, generator_dur_loss=1.477, generator_adv_loss=2.228, generator_feat_match_loss=4.527, over 1958.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 04:39:01,725 INFO [train.py:845] (2/4) Start epoch 735
2024-02-24 04:42:30,140 INFO [train.py:845] (2/4) Start epoch 736
2024-02-24 04:43:14,999 INFO [train.py:471] (2/4) Epoch 736, batch 5, global_batch_idx: 27200, batch size: 65, loss[discriminator_loss=2.426, discriminator_real_loss=1.188, discriminator_fake_loss=1.239, generator_loss=30.93, generator_mel_loss=20.1, generator_kl_loss=1.958, generator_dur_loss=1.465, generator_adv_loss=2.193, generator_feat_match_loss=5.219, over 65.00 samples.], tot_loss[discriminator_loss=2.609, discriminator_real_loss=1.385, discriminator_fake_loss=1.225, generator_loss=29.89, generator_mel_loss=19.8, generator_kl_loss=1.899, generator_dur_loss=1.479, generator_adv_loss=2.272, generator_feat_match_loss=4.436, over 436.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 04:43:15,000 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 04:43:24,063 INFO [train.py:534] (2/4) Epoch 736, validation: discriminator_loss=2.546, discriminator_real_loss=1.136, discriminator_fake_loss=1.41, generator_loss=30.93, generator_mel_loss=20.71, generator_kl_loss=2.152, generator_dur_loss=1.473, generator_adv_loss=1.866, generator_feat_match_loss=4.735, over 100.00 samples.
2024-02-24 04:43:24,064 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 04:46:09,408 INFO [train.py:845] (2/4) Start epoch 737
2024-02-24 04:48:08,520 INFO [train.py:471] (2/4) Epoch 737, batch 18, global_batch_idx: 27250, batch size: 51, loss[discriminator_loss=2.535, discriminator_real_loss=1.374, discriminator_fake_loss=1.162, generator_loss=31.06, generator_mel_loss=20.11, generator_kl_loss=1.974, generator_dur_loss=1.473, generator_adv_loss=2.438, generator_feat_match_loss=5.066, over 51.00 samples.], tot_loss[discriminator_loss=2.596, discriminator_real_loss=1.327, discriminator_fake_loss=1.269, generator_loss=30.75, generator_mel_loss=20.27, generator_kl_loss=1.963, generator_dur_loss=1.471, generator_adv_loss=2.295, generator_feat_match_loss=4.753, over 1562.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 04:49:41,128 INFO [train.py:845] (2/4) Start epoch 738
2024-02-24 04:52:38,543 INFO [train.py:471] (2/4) Epoch 738, batch 31, global_batch_idx: 27300, batch size: 69, loss[discriminator_loss=2.561, discriminator_real_loss=1.223, discriminator_fake_loss=1.338, generator_loss=30.79, generator_mel_loss=20.4, generator_kl_loss=1.862, generator_dur_loss=1.454, generator_adv_loss=2.41, generator_feat_match_loss=4.664, over 69.00 samples.], tot_loss[discriminator_loss=2.55, discriminator_real_loss=1.291, discriminator_fake_loss=1.259, generator_loss=30.34, generator_mel_loss=20.11, generator_kl_loss=1.962, generator_dur_loss=1.473, generator_adv_loss=2.221, generator_feat_match_loss=4.572, over 2262.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 04:53:05,651 INFO [train.py:845] (2/4) Start epoch 739
2024-02-24 04:56:35,436 INFO [train.py:845] (2/4) Start epoch 740
2024-02-24 04:57:22,315 INFO [train.py:471] (2/4) Epoch 740, batch 7, global_batch_idx: 27350, batch size: 55, loss[discriminator_loss=2.537, discriminator_real_loss=1.354, discriminator_fake_loss=1.183, generator_loss=30.66, generator_mel_loss=20.23, generator_kl_loss=1.888, generator_dur_loss=1.485, generator_adv_loss=2.492, generator_feat_match_loss=4.566, over 55.00 samples.], tot_loss[discriminator_loss=2.55, discriminator_real_loss=1.293, discriminator_fake_loss=1.257, generator_loss=30.2, generator_mel_loss=20.02, generator_kl_loss=1.928, generator_dur_loss=1.473, generator_adv_loss=2.266, generator_feat_match_loss=4.505, over 586.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:00:04,889 INFO [train.py:845] (2/4) Start epoch 741
2024-02-24 05:02:06,949 INFO [train.py:471] (2/4) Epoch 741, batch 20, global_batch_idx: 27400, batch size: 52, loss[discriminator_loss=2.527, discriminator_real_loss=1.269, discriminator_fake_loss=1.259, generator_loss=30.46, generator_mel_loss=20.16, generator_kl_loss=1.838, generator_dur_loss=1.478, generator_adv_loss=2.295, generator_feat_match_loss=4.688, over 52.00 samples.], tot_loss[discriminator_loss=2.516, discriminator_real_loss=1.263, discriminator_fake_loss=1.253, generator_loss=30.54, generator_mel_loss=19.9, generator_kl_loss=1.909, generator_dur_loss=1.47, generator_adv_loss=2.346, generator_feat_match_loss=4.917, over 1719.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:02:06,950 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 05:02:14,552 INFO [train.py:534] (2/4) Epoch 741, validation: discriminator_loss=2.579, discriminator_real_loss=1.24, discriminator_fake_loss=1.338, generator_loss=30.43, generator_mel_loss=20.39, generator_kl_loss=2.015, generator_dur_loss=1.472, generator_adv_loss=2.07, generator_feat_match_loss=4.485, over 100.00 samples.
2024-02-24 05:02:14,553 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 05:03:42,547 INFO [train.py:845] (2/4) Start epoch 742
2024-02-24 05:06:54,792 INFO [train.py:471] (2/4) Epoch 742, batch 33, global_batch_idx: 27450, batch size: 64, loss[discriminator_loss=2.641, discriminator_real_loss=1.328, discriminator_fake_loss=1.312, generator_loss=29.92, generator_mel_loss=19.84, generator_kl_loss=1.911, generator_dur_loss=1.483, generator_adv_loss=2.301, generator_feat_match_loss=4.379, over 64.00 samples.], tot_loss[discriminator_loss=2.53, discriminator_real_loss=1.275, discriminator_fake_loss=1.255, generator_loss=30.74, generator_mel_loss=20.22, generator_kl_loss=1.936, generator_dur_loss=1.472, generator_adv_loss=2.334, generator_feat_match_loss=4.776, over 2638.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:07:12,823 INFO [train.py:845] (2/4) Start epoch 743
2024-02-24 05:10:45,640 INFO [train.py:845] (2/4) Start epoch 744
2024-02-24 05:11:45,965 INFO [train.py:471] (2/4) Epoch 744, batch 9, global_batch_idx: 27500, batch size: 81, loss[discriminator_loss=2.59, discriminator_real_loss=1.459, discriminator_fake_loss=1.132, generator_loss=31.1, generator_mel_loss=20.24, generator_kl_loss=1.961, generator_dur_loss=1.471, generator_adv_loss=2.516, generator_feat_match_loss=4.906, over 81.00 samples.], tot_loss[discriminator_loss=2.569, discriminator_real_loss=1.294, discriminator_fake_loss=1.275, generator_loss=30.7, generator_mel_loss=20.23, generator_kl_loss=1.923, generator_dur_loss=1.48, generator_adv_loss=2.328, generator_feat_match_loss=4.737, over 699.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:14:15,426 INFO [train.py:845] (2/4) Start epoch 745
2024-02-24 05:16:27,696 INFO [train.py:471] (2/4) Epoch 745, batch 22, global_batch_idx: 27550, batch size: 126, loss[discriminator_loss=2.566, discriminator_real_loss=1.178, discriminator_fake_loss=1.389, generator_loss=30.39, generator_mel_loss=20.48, generator_kl_loss=1.998, generator_dur_loss=1.458, generator_adv_loss=2.098, generator_feat_match_loss=4.355, over 126.00 samples.], tot_loss[discriminator_loss=2.551, discriminator_real_loss=1.274, discriminator_fake_loss=1.277, generator_loss=30.46, generator_mel_loss=20.08, generator_kl_loss=1.959, generator_dur_loss=1.471, generator_adv_loss=2.3, generator_feat_match_loss=4.641, over 1753.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 4.0
2024-02-24 05:17:40,922 INFO [train.py:845] (2/4) Start epoch 746
2024-02-24 05:20:59,506 INFO [train.py:471] (2/4) Epoch 746, batch 35, global_batch_idx: 27600, batch size: 101, loss[discriminator_loss=2.668, discriminator_real_loss=1.152, discriminator_fake_loss=1.517, generator_loss=30.18, generator_mel_loss=20.04, generator_kl_loss=1.986, generator_dur_loss=1.465, generator_adv_loss=2.125, generator_feat_match_loss=4.562, over 101.00 samples.], tot_loss[discriminator_loss=2.513, discriminator_real_loss=1.265, discriminator_fake_loss=1.248, generator_loss=30.43, generator_mel_loss=19.87, generator_kl_loss=1.926, generator_dur_loss=1.474, generator_adv_loss=2.33, generator_feat_match_loss=4.824, over 2534.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:20:59,507 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 05:21:08,491 INFO [train.py:534] (2/4) Epoch 746, validation: discriminator_loss=2.729, discriminator_real_loss=1.148, discriminator_fake_loss=1.581, generator_loss=29.59, generator_mel_loss=20.25, generator_kl_loss=2.084, generator_dur_loss=1.47, generator_adv_loss=1.658, generator_feat_match_loss=4.128, over 100.00 samples.
2024-02-24 05:21:08,492 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 05:21:15,449 INFO [train.py:845] (2/4) Start epoch 747
2024-02-24 05:24:37,165 INFO [train.py:845] (2/4) Start epoch 748
2024-02-24 05:25:53,292 INFO [train.py:471] (2/4) Epoch 748, batch 11, global_batch_idx: 27650, batch size: 56, loss[discriminator_loss=2.463, discriminator_real_loss=1.295, discriminator_fake_loss=1.168, generator_loss=31.33, generator_mel_loss=20.17, generator_kl_loss=1.902, generator_dur_loss=1.48, generator_adv_loss=2.617, generator_feat_match_loss=5.156, over 56.00 samples.], tot_loss[discriminator_loss=2.543, discriminator_real_loss=1.29, discriminator_fake_loss=1.253, generator_loss=30.95, generator_mel_loss=20.3, generator_kl_loss=1.949, generator_dur_loss=1.468, generator_adv_loss=2.378, generator_feat_match_loss=4.855, over 1040.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 4.0
2024-02-24 05:28:09,817 INFO [train.py:845] (2/4) Start epoch 749
2024-02-24 05:30:39,189 INFO [train.py:471] (2/4) Epoch 749, batch 24, global_batch_idx: 27700, batch size: 67, loss[discriminator_loss=2.492, discriminator_real_loss=1.268, discriminator_fake_loss=1.225, generator_loss=30.56, generator_mel_loss=20.29, generator_kl_loss=1.939, generator_dur_loss=1.493, generator_adv_loss=2.256, generator_feat_match_loss=4.582, over 67.00 samples.], tot_loss[discriminator_loss=2.555, discriminator_real_loss=1.287, discriminator_fake_loss=1.268, generator_loss=30.23, generator_mel_loss=19.96, generator_kl_loss=1.927, generator_dur_loss=1.471, generator_adv_loss=2.263, generator_feat_match_loss=4.612, over 1940.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:31:41,632 INFO [train.py:845] (2/4) Start epoch 750
2024-02-24 05:35:09,996 INFO [train.py:845] (2/4) Start epoch 751
2024-02-24 05:35:23,455 INFO [train.py:471] (2/4) Epoch 751, batch 0, global_batch_idx: 27750, batch size: 95, loss[discriminator_loss=2.426, discriminator_real_loss=1.277, discriminator_fake_loss=1.148, generator_loss=31.07, generator_mel_loss=20.15, generator_kl_loss=1.845, generator_dur_loss=1.456, generator_adv_loss=2.494, generator_feat_match_loss=5.121, over 95.00 samples.], tot_loss[discriminator_loss=2.426, discriminator_real_loss=1.277, discriminator_fake_loss=1.148, generator_loss=31.07, generator_mel_loss=20.15, generator_kl_loss=1.845, generator_dur_loss=1.456, generator_adv_loss=2.494, generator_feat_match_loss=5.121, over 95.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 4.0
2024-02-24 05:38:34,818 INFO [train.py:845] (2/4) Start epoch 752
2024-02-24 05:39:58,363 INFO [train.py:471] (2/4) Epoch 752, batch 13, global_batch_idx: 27800, batch size: 52, loss[discriminator_loss=2.445, discriminator_real_loss=1.306, discriminator_fake_loss=1.141, generator_loss=29.9, generator_mel_loss=19.47, generator_kl_loss=1.886, generator_dur_loss=1.49, generator_adv_loss=2.322, generator_feat_match_loss=4.727, over 52.00 samples.], tot_loss[discriminator_loss=2.548, discriminator_real_loss=1.296, discriminator_fake_loss=1.252, generator_loss=30.08, generator_mel_loss=19.8, generator_kl_loss=1.933, generator_dur_loss=1.473, generator_adv_loss=2.263, generator_feat_match_loss=4.608, over 1006.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:39:58,365 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 05:40:06,987 INFO [train.py:534] (2/4) Epoch 752, validation: discriminator_loss=2.539, discriminator_real_loss=1.153, discriminator_fake_loss=1.386, generator_loss=30.33, generator_mel_loss=20.37, generator_kl_loss=1.985, generator_dur_loss=1.475, generator_adv_loss=1.972, generator_feat_match_loss=4.532, over 100.00 samples.
2024-02-24 05:40:06,988 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 05:42:13,988 INFO [train.py:845] (2/4) Start epoch 753
2024-02-24 05:44:47,656 INFO [train.py:471] (2/4) Epoch 753, batch 26, global_batch_idx: 27850, batch size: 59, loss[discriminator_loss=2.598, discriminator_real_loss=1.444, discriminator_fake_loss=1.153, generator_loss=30.09, generator_mel_loss=20.11, generator_kl_loss=1.909, generator_dur_loss=1.484, generator_adv_loss=2.449, generator_feat_match_loss=4.141, over 59.00 samples.], tot_loss[discriminator_loss=2.557, discriminator_real_loss=1.288, discriminator_fake_loss=1.269, generator_loss=30.47, generator_mel_loss=20.09, generator_kl_loss=1.937, generator_dur_loss=1.475, generator_adv_loss=2.308, generator_feat_match_loss=4.656, over 2075.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:45:43,231 INFO [train.py:845] (2/4) Start epoch 754
2024-02-24 05:49:18,185 INFO [train.py:845] (2/4) Start epoch 755
2024-02-24 05:49:45,231 INFO [train.py:471] (2/4) Epoch 755, batch 2, global_batch_idx: 27900, batch size: 101, loss[discriminator_loss=2.324, discriminator_real_loss=1.115, discriminator_fake_loss=1.21, generator_loss=31.74, generator_mel_loss=20.18, generator_kl_loss=1.953, generator_dur_loss=1.439, generator_adv_loss=2.43, generator_feat_match_loss=5.738, over 101.00 samples.], tot_loss[discriminator_loss=2.365, discriminator_real_loss=1.216, discriminator_fake_loss=1.149, generator_loss=31.54, generator_mel_loss=20.15, generator_kl_loss=1.979, generator_dur_loss=1.464, generator_adv_loss=2.44, generator_feat_match_loss=5.504, over 215.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:52:48,918 INFO [train.py:845] (2/4) Start epoch 756
2024-02-24 05:54:23,586 INFO [train.py:471] (2/4) Epoch 756, batch 15, global_batch_idx: 27950, batch size: 69, loss[discriminator_loss=2.559, discriminator_real_loss=1.377, discriminator_fake_loss=1.183, generator_loss=30.71, generator_mel_loss=20.44, generator_kl_loss=1.973, generator_dur_loss=1.458, generator_adv_loss=2.188, generator_feat_match_loss=4.648, over 69.00 samples.], tot_loss[discriminator_loss=2.547, discriminator_real_loss=1.275, discriminator_fake_loss=1.272, generator_loss=30.26, generator_mel_loss=20.02, generator_kl_loss=1.943, generator_dur_loss=1.464, generator_adv_loss=2.233, generator_feat_match_loss=4.604, over 1316.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 4.0
2024-02-24 05:56:15,634 INFO [train.py:845] (2/4) Start epoch 757
2024-02-24 05:58:57,607 INFO [train.py:471] (2/4) Epoch 757, batch 28, global_batch_idx: 28000, batch size: 53, loss[discriminator_loss=2.605, discriminator_real_loss=1.12, discriminator_fake_loss=1.485, generator_loss=29.89, generator_mel_loss=20.08, generator_kl_loss=1.905, generator_dur_loss=1.487, generator_adv_loss=2.039, generator_feat_match_loss=4.383, over 53.00 samples.], tot_loss[discriminator_loss=2.529, discriminator_real_loss=1.272, discriminator_fake_loss=1.257, generator_loss=30.56, generator_mel_loss=20.08, generator_kl_loss=1.927, generator_dur_loss=1.47, generator_adv_loss=2.342, generator_feat_match_loss=4.739, over 2071.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 05:58:57,609 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 05:59:05,928 INFO [train.py:534] (2/4) Epoch 757, validation: discriminator_loss=2.549, discriminator_real_loss=1.255, discriminator_fake_loss=1.294, generator_loss=30.91, generator_mel_loss=20.79, generator_kl_loss=2.058, generator_dur_loss=1.47, generator_adv_loss=2.022, generator_feat_match_loss=4.573, over 100.00 samples.
2024-02-24 05:59:05,929 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 05:59:55,157 INFO [train.py:845] (2/4) Start epoch 758
2024-02-24 06:03:20,294 INFO [train.py:845] (2/4) Start epoch 759
2024-02-24 06:03:54,963 INFO [train.py:471] (2/4) Epoch 759, batch 4, global_batch_idx: 28050, batch size: 153, loss[discriminator_loss=2.496, discriminator_real_loss=1.203, discriminator_fake_loss=1.294, generator_loss=30.84, generator_mel_loss=20.22, generator_kl_loss=2.017, generator_dur_loss=1.438, generator_adv_loss=2.199, generator_feat_match_loss=4.961, over 153.00 samples.], tot_loss[discriminator_loss=2.526, discriminator_real_loss=1.244, discriminator_fake_loss=1.283, generator_loss=30.57, generator_mel_loss=20.07, generator_kl_loss=1.975, generator_dur_loss=1.464, generator_adv_loss=2.253, generator_feat_match_loss=4.807, over 379.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:06:50,742 INFO [train.py:845] (2/4) Start epoch 760
2024-02-24 06:08:47,634 INFO [train.py:471] (2/4) Epoch 760, batch 17, global_batch_idx: 28100, batch size: 126, loss[discriminator_loss=2.641, discriminator_real_loss=1.222, discriminator_fake_loss=1.418, generator_loss=30.09, generator_mel_loss=19.9, generator_kl_loss=1.963, generator_dur_loss=1.454, generator_adv_loss=2.104, generator_feat_match_loss=4.672, over 126.00 samples.], tot_loss[discriminator_loss=2.543, discriminator_real_loss=1.271, discriminator_fake_loss=1.272, generator_loss=30.49, generator_mel_loss=19.97, generator_kl_loss=1.954, generator_dur_loss=1.469, generator_adv_loss=2.32, generator_feat_match_loss=4.779, over 1378.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:10:22,944 INFO [train.py:845] (2/4) Start epoch 761
2024-02-24 06:13:13,181 INFO [train.py:471] (2/4) Epoch 761, batch 30, global_batch_idx: 28150, batch size: 56, loss[discriminator_loss=2.539, discriminator_real_loss=1.256, discriminator_fake_loss=1.282, generator_loss=30.16, generator_mel_loss=19.93, generator_kl_loss=2.018, generator_dur_loss=1.459, generator_adv_loss=2.273, generator_feat_match_loss=4.48, over 56.00 samples.], tot_loss[discriminator_loss=2.534, discriminator_real_loss=1.272, discriminator_fake_loss=1.262, generator_loss=30.36, generator_mel_loss=19.91, generator_kl_loss=1.942, generator_dur_loss=1.469, generator_adv_loss=2.292, generator_feat_match_loss=4.754, over 2211.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:13:50,478 INFO [train.py:845] (2/4) Start epoch 762
2024-02-24 06:17:20,488 INFO [train.py:845] (2/4) Start epoch 763
2024-02-24 06:18:12,365 INFO [train.py:471] (2/4) Epoch 763, batch 6, global_batch_idx: 28200, batch size: 71, loss[discriminator_loss=2.527, discriminator_real_loss=1.233, discriminator_fake_loss=1.293, generator_loss=30.22, generator_mel_loss=19.67, generator_kl_loss=1.84, generator_dur_loss=1.48, generator_adv_loss=2.211, generator_feat_match_loss=5.023, over 71.00 samples.], tot_loss[discriminator_loss=2.55, discriminator_real_loss=1.253, discriminator_fake_loss=1.296, generator_loss=30.43, generator_mel_loss=19.94, generator_kl_loss=1.952, generator_dur_loss=1.482, generator_adv_loss=2.325, generator_feat_match_loss=4.731, over 520.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:18:12,367 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 06:18:21,439 INFO [train.py:534] (2/4) Epoch 763, validation: discriminator_loss=2.717, discriminator_real_loss=1.137, discriminator_fake_loss=1.581, generator_loss=30.59, generator_mel_loss=20.96, generator_kl_loss=1.983, generator_dur_loss=1.474, generator_adv_loss=1.719, generator_feat_match_loss=4.459, over 100.00 samples.
2024-02-24 06:18:21,440 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 06:21:00,352 INFO [train.py:845] (2/4) Start epoch 764
2024-02-24 06:22:51,529 INFO [train.py:471] (2/4) Epoch 764, batch 19, global_batch_idx: 28250, batch size: 60, loss[discriminator_loss=2.477, discriminator_real_loss=1.4, discriminator_fake_loss=1.076, generator_loss=30.63, generator_mel_loss=19.75, generator_kl_loss=2.039, generator_dur_loss=1.452, generator_adv_loss=2.572, generator_feat_match_loss=4.824, over 60.00 samples.], tot_loss[discriminator_loss=2.51, discriminator_real_loss=1.262, discriminator_fake_loss=1.248, generator_loss=30.48, generator_mel_loss=19.97, generator_kl_loss=1.961, generator_dur_loss=1.467, generator_adv_loss=2.325, generator_feat_match_loss=4.758, over 1428.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 4.0
2024-02-24 06:24:26,669 INFO [train.py:845] (2/4) Start epoch 765
2024-02-24 06:27:35,219 INFO [train.py:471] (2/4) Epoch 765, batch 32, global_batch_idx: 28300, batch size: 73, loss[discriminator_loss=2.492, discriminator_real_loss=1.163, discriminator_fake_loss=1.33, generator_loss=31.23, generator_mel_loss=20.04, generator_kl_loss=2.115, generator_dur_loss=1.469, generator_adv_loss=2.32, generator_feat_match_loss=5.285, over 73.00 samples.], tot_loss[discriminator_loss=2.539, discriminator_real_loss=1.282, discriminator_fake_loss=1.257, generator_loss=30.57, generator_mel_loss=20.03, generator_kl_loss=1.952, generator_dur_loss=1.465, generator_adv_loss=2.337, generator_feat_match_loss=4.786, over 2610.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:27:53,239 INFO [train.py:845] (2/4) Start epoch 766
2024-02-24 06:31:14,468 INFO [train.py:845] (2/4) Start epoch 767
2024-02-24 06:32:07,230 INFO [train.py:471] (2/4) Epoch 767, batch 8, global_batch_idx: 28350, batch size: 67, loss[discriminator_loss=2.484, discriminator_real_loss=1.31, discriminator_fake_loss=1.174, generator_loss=30.35, generator_mel_loss=19.9, generator_kl_loss=1.929, generator_dur_loss=1.477, generator_adv_loss=2.355, generator_feat_match_loss=4.691, over 67.00 samples.], tot_loss[discriminator_loss=2.503, discriminator_real_loss=1.26, discriminator_fake_loss=1.243, generator_loss=30.32, generator_mel_loss=19.93, generator_kl_loss=1.949, generator_dur_loss=1.475, generator_adv_loss=2.271, generator_feat_match_loss=4.693, over 633.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:34:41,227 INFO [train.py:845] (2/4) Start epoch 768
2024-02-24 06:36:45,670 INFO [train.py:471] (2/4) Epoch 768, batch 21, global_batch_idx: 28400, batch size: 58, loss[discriminator_loss=2.404, discriminator_real_loss=1.352, discriminator_fake_loss=1.053, generator_loss=30.77, generator_mel_loss=19.57, generator_kl_loss=1.973, generator_dur_loss=1.48, generator_adv_loss=2.43, generator_feat_match_loss=5.32, over 58.00 samples.], tot_loss[discriminator_loss=2.483, discriminator_real_loss=1.251, discriminator_fake_loss=1.232, generator_loss=30.57, generator_mel_loss=19.93, generator_kl_loss=1.965, generator_dur_loss=1.469, generator_adv_loss=2.335, generator_feat_match_loss=4.877, over 1689.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:36:45,671 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 06:36:54,935 INFO [train.py:534] (2/4) Epoch 768, validation: discriminator_loss=2.375, discriminator_real_loss=1.125, discriminator_fake_loss=1.25, generator_loss=32.21, generator_mel_loss=20.8, generator_kl_loss=2.137, generator_dur_loss=1.47, generator_adv_loss=2.258, generator_feat_match_loss=5.54, over 100.00 samples.
2024-02-24 06:36:54,937 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 06:38:21,905 INFO [train.py:845] (2/4) Start epoch 769
2024-02-24 06:41:47,270 INFO [train.py:471] (2/4) Epoch 769, batch 34, global_batch_idx: 28450, batch size: 61, loss[discriminator_loss=2.393, discriminator_real_loss=1.251, discriminator_fake_loss=1.142, generator_loss=30.31, generator_mel_loss=19.52, generator_kl_loss=1.899, generator_dur_loss=1.468, generator_adv_loss=2.33, generator_feat_match_loss=5.086, over 61.00 samples.], tot_loss[discriminator_loss=2.447, discriminator_real_loss=1.234, discriminator_fake_loss=1.213, generator_loss=30.82, generator_mel_loss=19.82, generator_kl_loss=1.95, generator_dur_loss=1.472, generator_adv_loss=2.438, generator_feat_match_loss=5.14, over 2589.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 4.0
2024-02-24 06:41:55,844 INFO [train.py:845] (2/4) Start epoch 770
2024-02-24 06:45:26,949 INFO [train.py:845] (2/4) Start epoch 771
2024-02-24 06:46:30,342 INFO [train.py:471] (2/4) Epoch 771, batch 10, global_batch_idx: 28500, batch size: 61, loss[discriminator_loss=2.492, discriminator_real_loss=1.257, discriminator_fake_loss=1.234, generator_loss=31.84, generator_mel_loss=20.87, generator_kl_loss=1.949, generator_dur_loss=1.449, generator_adv_loss=2.314, generator_feat_match_loss=5.254, over 61.00 samples.], tot_loss[discriminator_loss=2.587, discriminator_real_loss=1.31, discriminator_fake_loss=1.277, generator_loss=30.91, generator_mel_loss=20.52, generator_kl_loss=1.985, generator_dur_loss=1.471, generator_adv_loss=2.257, generator_feat_match_loss=4.674, over 742.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:48:58,708 INFO [train.py:845] (2/4) Start epoch 772
2024-02-24 06:51:20,961 INFO [train.py:471] (2/4) Epoch 772, batch 23, global_batch_idx: 28550, batch size: 90, loss[discriminator_loss=2.289, discriminator_real_loss=1.082, discriminator_fake_loss=1.208, generator_loss=31.18, generator_mel_loss=19.84, generator_kl_loss=1.915, generator_dur_loss=1.477, generator_adv_loss=2.527, generator_feat_match_loss=5.414, over 90.00 samples.], tot_loss[discriminator_loss=2.541, discriminator_real_loss=1.264, discriminator_fake_loss=1.276, generator_loss=30.33, generator_mel_loss=19.69, generator_kl_loss=1.901, generator_dur_loss=1.469, generator_adv_loss=2.343, generator_feat_match_loss=4.923, over 1868.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:52:27,882 INFO [train.py:845] (2/4) Start epoch 773
2024-02-24 06:56:02,128 INFO [train.py:471] (2/4) Epoch 773, batch 36, global_batch_idx: 28600, batch size: 67, loss[discriminator_loss=2.562, discriminator_real_loss=1.403, discriminator_fake_loss=1.159, generator_loss=30.71, generator_mel_loss=19.96, generator_kl_loss=1.936, generator_dur_loss=1.476, generator_adv_loss=2.41, generator_feat_match_loss=4.926, over 67.00 samples.], tot_loss[discriminator_loss=2.572, discriminator_real_loss=1.303, discriminator_fake_loss=1.27, generator_loss=30.27, generator_mel_loss=20.12, generator_kl_loss=1.958, generator_dur_loss=1.469, generator_adv_loss=2.187, generator_feat_match_loss=4.536, over 2776.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 06:56:02,130 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 06:56:10,326 INFO [train.py:534] (2/4) Epoch 773, validation: discriminator_loss=2.535, discriminator_real_loss=1.156, discriminator_fake_loss=1.379, generator_loss=31.03, generator_mel_loss=20.73, generator_kl_loss=2.099, generator_dur_loss=1.477, generator_adv_loss=1.989, generator_feat_match_loss=4.741, over 100.00 samples.
2024-02-24 06:56:10,328 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 06:56:10,824 INFO [train.py:845] (2/4) Start epoch 774
2024-02-24 06:59:41,282 INFO [train.py:845] (2/4) Start epoch 775
2024-02-24 07:00:54,844 INFO [train.py:471] (2/4) Epoch 775, batch 12, global_batch_idx: 28650, batch size: 52, loss[discriminator_loss=2.547, discriminator_real_loss=1.281, discriminator_fake_loss=1.266, generator_loss=29.31, generator_mel_loss=19.32, generator_kl_loss=1.917, generator_dur_loss=1.46, generator_adv_loss=2.365, generator_feat_match_loss=4.254, over 52.00 samples.], tot_loss[discriminator_loss=2.52, discriminator_real_loss=1.25, discriminator_fake_loss=1.27, generator_loss=30.32, generator_mel_loss=19.77, generator_kl_loss=1.921, generator_dur_loss=1.475, generator_adv_loss=2.326, generator_feat_match_loss=4.829, over 887.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 4.0
2024-02-24 07:03:11,846 INFO [train.py:845] (2/4) Start epoch 776
2024-02-24 07:05:41,339 INFO [train.py:471] (2/4) Epoch 776, batch 25, global_batch_idx: 28700, batch size: 55, loss[discriminator_loss=2.281, discriminator_real_loss=1.046, discriminator_fake_loss=1.235, generator_loss=31.38, generator_mel_loss=19.8, generator_kl_loss=1.856, generator_dur_loss=1.521, generator_adv_loss=2.566, generator_feat_match_loss=5.633, over 55.00 samples.], tot_loss[discriminator_loss=2.476, discriminator_real_loss=1.236, discriminator_fake_loss=1.24, generator_loss=30.69, generator_mel_loss=19.9, generator_kl_loss=1.934, generator_dur_loss=1.466, generator_adv_loss=2.37, generator_feat_match_loss=5.019, over 2005.00 samples.], cur_lr_g: 1.82e-04, cur_lr_d: 1.82e-04, grad_scale: 8.0
2024-02-24 07:06:39,834 INFO [train.py:845] (2/4) Start epoch 777
2024-02-24 07:10:13,439 INFO [train.py:845] (2/4) Start epoch 778
2024-02-24 07:10:30,476 INFO [train.py:471] (2/4) Epoch 778, batch 1, global_batch_idx: 28750, batch size: 52, loss[discriminator_loss=2.404, discriminator_real_loss=1.123, discriminator_fake_loss=1.281, generator_loss=31.11, generator_mel_loss=20.14, generator_kl_loss=1.957, generator_dur_loss=1.495, generator_adv_loss=2.438, generator_feat_match_loss=5.074, over 52.00 samples.], tot_loss[discriminator_loss=2.418, discriminator_real_loss=1.217, discriminator_fake_loss=1.201, generator_loss=30.48, generator_mel_loss=19.72, generator_kl_loss=1.944, generator_dur_loss=1.474, generator_adv_loss=2.344, generator_feat_match_loss=4.998, over 117.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:13:45,592 INFO [train.py:845] (2/4) Start epoch 779
2024-02-24 07:15:16,327 INFO [train.py:471] (2/4) Epoch 779, batch 14, global_batch_idx: 28800, batch size: 101, loss[discriminator_loss=2.582, discriminator_real_loss=1.264, discriminator_fake_loss=1.319, generator_loss=30.55, generator_mel_loss=20.25, generator_kl_loss=1.943, generator_dur_loss=1.457, generator_adv_loss=2.168, generator_feat_match_loss=4.738, over 101.00 samples.], tot_loss[discriminator_loss=2.515, discriminator_real_loss=1.282, discriminator_fake_loss=1.233, generator_loss=30.52, generator_mel_loss=20.03, generator_kl_loss=1.943, generator_dur_loss=1.466, generator_adv_loss=2.297, generator_feat_match_loss=4.79, over 1216.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 16.0
2024-02-24 07:15:16,329 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 07:15:25,584 INFO [train.py:534] (2/4) Epoch 779, validation: discriminator_loss=2.589, discriminator_real_loss=1.221, discriminator_fake_loss=1.368, generator_loss=30.77, generator_mel_loss=20.74, generator_kl_loss=2.139, generator_dur_loss=1.47, generator_adv_loss=2.005, generator_feat_match_loss=4.42, over 100.00 samples.
2024-02-24 07:15:25,585 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 07:17:17,149 INFO [train.py:845] (2/4) Start epoch 780
2024-02-24 07:19:47,146 INFO [train.py:471] (2/4) Epoch 780, batch 27, global_batch_idx: 28850, batch size: 110, loss[discriminator_loss=2.379, discriminator_real_loss=1.234, discriminator_fake_loss=1.144, generator_loss=31.53, generator_mel_loss=20.01, generator_kl_loss=2.026, generator_dur_loss=1.47, generator_adv_loss=2.451, generator_feat_match_loss=5.574, over 110.00 samples.], tot_loss[discriminator_loss=2.462, discriminator_real_loss=1.252, discriminator_fake_loss=1.21, generator_loss=30.43, generator_mel_loss=19.59, generator_kl_loss=1.934, generator_dur_loss=1.472, generator_adv_loss=2.403, generator_feat_match_loss=5.029, over 1955.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 4.0
2024-02-24 07:20:41,232 INFO [train.py:845] (2/4) Start epoch 781
2024-02-24 07:24:09,060 INFO [train.py:845] (2/4) Start epoch 782
2024-02-24 07:24:37,478 INFO [train.py:471] (2/4) Epoch 782, batch 3, global_batch_idx: 28900, batch size: 69, loss[discriminator_loss=2.586, discriminator_real_loss=1.212, discriminator_fake_loss=1.375, generator_loss=29.24, generator_mel_loss=19.29, generator_kl_loss=1.928, generator_dur_loss=1.465, generator_adv_loss=2.195, generator_feat_match_loss=4.363, over 69.00 samples.], tot_loss[discriminator_loss=2.563, discriminator_real_loss=1.271, discriminator_fake_loss=1.293, generator_loss=29.85, generator_mel_loss=19.69, generator_kl_loss=1.913, generator_dur_loss=1.479, generator_adv_loss=2.172, generator_feat_match_loss=4.588, over 255.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:27:33,236 INFO [train.py:845] (2/4) Start epoch 783
2024-02-24 07:29:08,288 INFO [train.py:471] (2/4) Epoch 783, batch 16, global_batch_idx: 28950, batch size: 110, loss[discriminator_loss=2.562, discriminator_real_loss=1.395, discriminator_fake_loss=1.168, generator_loss=31.07, generator_mel_loss=20.16, generator_kl_loss=2.001, generator_dur_loss=1.476, generator_adv_loss=2.537, generator_feat_match_loss=4.891, over 110.00 samples.], tot_loss[discriminator_loss=2.543, discriminator_real_loss=1.285, discriminator_fake_loss=1.258, generator_loss=30.7, generator_mel_loss=20.07, generator_kl_loss=1.978, generator_dur_loss=1.469, generator_adv_loss=2.369, generator_feat_match_loss=4.813, over 1338.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:31:02,582 INFO [train.py:845] (2/4) Start epoch 784
2024-02-24 07:33:45,319 INFO [train.py:471] (2/4) Epoch 784, batch 29, global_batch_idx: 29000, batch size: 50, loss[discriminator_loss=2.598, discriminator_real_loss=1.185, discriminator_fake_loss=1.412, generator_loss=29.54, generator_mel_loss=19.22, generator_kl_loss=1.873, generator_dur_loss=1.486, generator_adv_loss=2.238, generator_feat_match_loss=4.719, over 50.00 samples.], tot_loss[discriminator_loss=2.468, discriminator_real_loss=1.24, discriminator_fake_loss=1.228, generator_loss=30.38, generator_mel_loss=19.61, generator_kl_loss=1.938, generator_dur_loss=1.474, generator_adv_loss=2.384, generator_feat_match_loss=4.974, over 2048.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:33:45,321 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 07:33:53,664 INFO [train.py:534] (2/4) Epoch 784, validation: discriminator_loss=2.572, discriminator_real_loss=1.211, discriminator_fake_loss=1.361, generator_loss=30.38, generator_mel_loss=20.35, generator_kl_loss=2.043, generator_dur_loss=1.468, generator_adv_loss=1.891, generator_feat_match_loss=4.631, over 100.00 samples.
2024-02-24 07:33:53,665 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 07:34:35,709 INFO [train.py:845] (2/4) Start epoch 785
2024-02-24 07:38:01,847 INFO [train.py:845] (2/4) Start epoch 786
2024-02-24 07:38:41,875 INFO [train.py:471] (2/4) Epoch 786, batch 5, global_batch_idx: 29050, batch size: 49, loss[discriminator_loss=2.732, discriminator_real_loss=1.253, discriminator_fake_loss=1.479, generator_loss=29.62, generator_mel_loss=19.67, generator_kl_loss=1.896, generator_dur_loss=1.464, generator_adv_loss=2.113, generator_feat_match_loss=4.477, over 49.00 samples.], tot_loss[discriminator_loss=2.466, discriminator_real_loss=1.229, discriminator_fake_loss=1.237, generator_loss=30.52, generator_mel_loss=19.75, generator_kl_loss=1.934, generator_dur_loss=1.478, generator_adv_loss=2.37, generator_feat_match_loss=4.988, over 347.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 4.0
2024-02-24 07:41:29,538 INFO [train.py:845] (2/4) Start epoch 787
2024-02-24 07:43:17,689 INFO [train.py:471] (2/4) Epoch 787, batch 18, global_batch_idx: 29100, batch size: 76, loss[discriminator_loss=2.676, discriminator_real_loss=1.177, discriminator_fake_loss=1.5, generator_loss=29.49, generator_mel_loss=19.61, generator_kl_loss=1.975, generator_dur_loss=1.464, generator_adv_loss=2.207, generator_feat_match_loss=4.23, over 76.00 samples.], tot_loss[discriminator_loss=2.494, discriminator_real_loss=1.231, discriminator_fake_loss=1.263, generator_loss=30.33, generator_mel_loss=19.67, generator_kl_loss=1.934, generator_dur_loss=1.465, generator_adv_loss=2.346, generator_feat_match_loss=4.912, over 1467.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:44:57,310 INFO [train.py:845] (2/4) Start epoch 788
2024-02-24 07:47:52,117 INFO [train.py:471] (2/4) Epoch 788, batch 31, global_batch_idx: 29150, batch size: 61, loss[discriminator_loss=2.52, discriminator_real_loss=1.369, discriminator_fake_loss=1.149, generator_loss=29.55, generator_mel_loss=19.38, generator_kl_loss=1.967, generator_dur_loss=1.486, generator_adv_loss=2.254, generator_feat_match_loss=4.457, over 61.00 samples.], tot_loss[discriminator_loss=2.564, discriminator_real_loss=1.298, discriminator_fake_loss=1.267, generator_loss=30.16, generator_mel_loss=20.02, generator_kl_loss=1.967, generator_dur_loss=1.471, generator_adv_loss=2.174, generator_feat_match_loss=4.527, over 2297.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:48:23,737 INFO [train.py:845] (2/4) Start epoch 789
2024-02-24 07:51:51,957 INFO [train.py:845] (2/4) Start epoch 790
2024-02-24 07:52:38,828 INFO [train.py:471] (2/4) Epoch 790, batch 7, global_batch_idx: 29200, batch size: 58, loss[discriminator_loss=2.516, discriminator_real_loss=1.418, discriminator_fake_loss=1.098, generator_loss=30.52, generator_mel_loss=19.37, generator_kl_loss=1.952, generator_dur_loss=1.478, generator_adv_loss=2.521, generator_feat_match_loss=5.199, over 58.00 samples.], tot_loss[discriminator_loss=2.493, discriminator_real_loss=1.259, discriminator_fake_loss=1.234, generator_loss=30.3, generator_mel_loss=19.41, generator_kl_loss=1.882, generator_dur_loss=1.471, generator_adv_loss=2.391, generator_feat_match_loss=5.146, over 499.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:52:38,829 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 07:52:47,615 INFO [train.py:534] (2/4) Epoch 790, validation: discriminator_loss=2.39, discriminator_real_loss=1.175, discriminator_fake_loss=1.215, generator_loss=31.85, generator_mel_loss=20.66, generator_kl_loss=2.072, generator_dur_loss=1.477, generator_adv_loss=2.236, generator_feat_match_loss=5.412, over 100.00 samples.
2024-02-24 07:52:47,616 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 07:55:33,609 INFO [train.py:845] (2/4) Start epoch 791
2024-02-24 07:57:35,638 INFO [train.py:471] (2/4) Epoch 791, batch 20, global_batch_idx: 29250, batch size: 51, loss[discriminator_loss=2.521, discriminator_real_loss=1.157, discriminator_fake_loss=1.364, generator_loss=30.4, generator_mel_loss=19.79, generator_kl_loss=1.883, generator_dur_loss=1.487, generator_adv_loss=2.279, generator_feat_match_loss=4.957, over 51.00 samples.], tot_loss[discriminator_loss=2.544, discriminator_real_loss=1.287, discriminator_fake_loss=1.257, generator_loss=30.31, generator_mel_loss=19.91, generator_kl_loss=1.935, generator_dur_loss=1.473, generator_adv_loss=2.278, generator_feat_match_loss=4.713, over 1508.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 07:59:06,490 INFO [train.py:845] (2/4) Start epoch 792
2024-02-24 08:02:24,098 INFO [train.py:471] (2/4) Epoch 792, batch 33, global_batch_idx: 29300, batch size: 54, loss[discriminator_loss=2.438, discriminator_real_loss=1.235, discriminator_fake_loss=1.201, generator_loss=30.83, generator_mel_loss=19.92, generator_kl_loss=1.971, generator_dur_loss=1.498, generator_adv_loss=2.309, generator_feat_match_loss=5.133, over 54.00 samples.], tot_loss[discriminator_loss=2.561, discriminator_real_loss=1.294, discriminator_fake_loss=1.267, generator_loss=30.16, generator_mel_loss=20.01, generator_kl_loss=1.951, generator_dur_loss=1.469, generator_adv_loss=2.173, generator_feat_match_loss=4.563, over 2505.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:02:41,657 INFO [train.py:845] (2/4) Start epoch 793
2024-02-24 08:06:04,606 INFO [train.py:845] (2/4) Start epoch 794
2024-02-24 08:07:02,484 INFO [train.py:471] (2/4) Epoch 794, batch 9, global_batch_idx: 29350, batch size: 55, loss[discriminator_loss=2.531, discriminator_real_loss=1.256, discriminator_fake_loss=1.274, generator_loss=29.63, generator_mel_loss=19.19, generator_kl_loss=1.939, generator_dur_loss=1.517, generator_adv_loss=2.287, generator_feat_match_loss=4.695, over 55.00 samples.], tot_loss[discriminator_loss=2.381, discriminator_real_loss=1.205, discriminator_fake_loss=1.175, generator_loss=30.62, generator_mel_loss=19.61, generator_kl_loss=1.929, generator_dur_loss=1.469, generator_adv_loss=2.468, generator_feat_match_loss=5.147, over 667.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:09:34,305 INFO [train.py:845] (2/4) Start epoch 795
2024-02-24 08:11:50,421 INFO [train.py:471] (2/4) Epoch 795, batch 22, global_batch_idx: 29400, batch size: 56, loss[discriminator_loss=2.465, discriminator_real_loss=1.199, discriminator_fake_loss=1.265, generator_loss=30.89, generator_mel_loss=20.33, generator_kl_loss=1.94, generator_dur_loss=1.494, generator_adv_loss=2.387, generator_feat_match_loss=4.738, over 56.00 samples.], tot_loss[discriminator_loss=2.526, discriminator_real_loss=1.261, discriminator_fake_loss=1.265, generator_loss=30.81, generator_mel_loss=20.26, generator_kl_loss=1.978, generator_dur_loss=1.464, generator_adv_loss=2.274, generator_feat_match_loss=4.839, over 1892.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:11:50,422 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 08:11:58,162 INFO [train.py:534] (2/4) Epoch 795, validation: discriminator_loss=2.464, discriminator_real_loss=1.165, discriminator_fake_loss=1.299, generator_loss=31.25, generator_mel_loss=20.94, generator_kl_loss=2.144, generator_dur_loss=1.465, generator_adv_loss=1.949, generator_feat_match_loss=4.756, over 100.00 samples.
2024-02-24 08:11:58,163 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 08:13:12,604 INFO [train.py:845] (2/4) Start epoch 796
2024-02-24 08:16:40,087 INFO [train.py:471] (2/4) Epoch 796, batch 35, global_batch_idx: 29450, batch size: 63, loss[discriminator_loss=2.424, discriminator_real_loss=1.404, discriminator_fake_loss=1.02, generator_loss=31.07, generator_mel_loss=19.96, generator_kl_loss=1.88, generator_dur_loss=1.465, generator_adv_loss=2.504, generator_feat_match_loss=5.254, over 63.00 samples.], tot_loss[discriminator_loss=2.494, discriminator_real_loss=1.26, discriminator_fake_loss=1.234, generator_loss=30.57, generator_mel_loss=19.72, generator_kl_loss=1.942, generator_dur_loss=1.466, generator_adv_loss=2.385, generator_feat_match_loss=5.055, over 2581.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:16:46,401 INFO [train.py:845] (2/4) Start epoch 797
2024-02-24 08:20:15,865 INFO [train.py:845] (2/4) Start epoch 798
2024-02-24 08:21:30,772 INFO [train.py:471] (2/4) Epoch 798, batch 11, global_batch_idx: 29500, batch size: 65, loss[discriminator_loss=2.49, discriminator_real_loss=1.223, discriminator_fake_loss=1.268, generator_loss=30.99, generator_mel_loss=20.5, generator_kl_loss=2.021, generator_dur_loss=1.469, generator_adv_loss=2.254, generator_feat_match_loss=4.75, over 65.00 samples.], tot_loss[discriminator_loss=2.539, discriminator_real_loss=1.27, discriminator_fake_loss=1.269, generator_loss=30.71, generator_mel_loss=20.14, generator_kl_loss=1.941, generator_dur_loss=1.459, generator_adv_loss=2.285, generator_feat_match_loss=4.886, over 941.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:23:44,870 INFO [train.py:845] (2/4) Start epoch 799
2024-02-24 08:26:02,721 INFO [train.py:471] (2/4) Epoch 799, batch 24, global_batch_idx: 29550, batch size: 69, loss[discriminator_loss=2.293, discriminator_real_loss=1.144, discriminator_fake_loss=1.15, generator_loss=31.12, generator_mel_loss=19.73, generator_kl_loss=1.88, generator_dur_loss=1.477, generator_adv_loss=2.486, generator_feat_match_loss=5.547, over 69.00 samples.], tot_loss[discriminator_loss=2.474, discriminator_real_loss=1.256, discriminator_fake_loss=1.218, generator_loss=30.76, generator_mel_loss=19.82, generator_kl_loss=1.932, generator_dur_loss=1.468, generator_adv_loss=2.417, generator_feat_match_loss=5.12, over 1677.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:27:10,655 INFO [train.py:845] (2/4) Start epoch 800
2024-02-24 08:30:35,330 INFO [train.py:845] (2/4) Start epoch 801
2024-02-24 08:30:48,712 INFO [train.py:471] (2/4) Epoch 801, batch 0, global_batch_idx: 29600, batch size: 60, loss[discriminator_loss=2.684, discriminator_real_loss=1.423, discriminator_fake_loss=1.261, generator_loss=30.36, generator_mel_loss=20.14, generator_kl_loss=2.028, generator_dur_loss=1.481, generator_adv_loss=2.189, generator_feat_match_loss=4.52, over 60.00 samples.], tot_loss[discriminator_loss=2.684, discriminator_real_loss=1.423, discriminator_fake_loss=1.261, generator_loss=30.36, generator_mel_loss=20.14, generator_kl_loss=2.028, generator_dur_loss=1.481, generator_adv_loss=2.189, generator_feat_match_loss=4.52, over 60.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:30:48,713 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 08:30:57,704 INFO [train.py:534] (2/4) Epoch 801, validation: discriminator_loss=2.63, discriminator_real_loss=1.412, discriminator_fake_loss=1.218, generator_loss=31.49, generator_mel_loss=20.82, generator_kl_loss=2.175, generator_dur_loss=1.469, generator_adv_loss=2.263, generator_feat_match_loss=4.765, over 100.00 samples.
2024-02-24 08:30:57,705 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 08:34:15,842 INFO [train.py:845] (2/4) Start epoch 802
2024-02-24 08:35:33,295 INFO [train.py:471] (2/4) Epoch 802, batch 13, global_batch_idx: 29650, batch size: 52, loss[discriminator_loss=2.609, discriminator_real_loss=1.271, discriminator_fake_loss=1.338, generator_loss=29.93, generator_mel_loss=19.96, generator_kl_loss=2.028, generator_dur_loss=1.459, generator_adv_loss=2.104, generator_feat_match_loss=4.375, over 52.00 samples.], tot_loss[discriminator_loss=2.571, discriminator_real_loss=1.288, discriminator_fake_loss=1.283, generator_loss=29.86, generator_mel_loss=19.87, generator_kl_loss=1.95, generator_dur_loss=1.467, generator_adv_loss=2.131, generator_feat_match_loss=4.447, over 893.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:37:49,148 INFO [train.py:845] (2/4) Start epoch 803
2024-02-24 08:40:19,926 INFO [train.py:471] (2/4) Epoch 803, batch 26, global_batch_idx: 29700, batch size: 50, loss[discriminator_loss=2.535, discriminator_real_loss=1.437, discriminator_fake_loss=1.1, generator_loss=31.01, generator_mel_loss=19.79, generator_kl_loss=2.034, generator_dur_loss=1.479, generator_adv_loss=2.465, generator_feat_match_loss=5.242, over 50.00 samples.], tot_loss[discriminator_loss=2.609, discriminator_real_loss=1.318, discriminator_fake_loss=1.291, generator_loss=30.48, generator_mel_loss=20.13, generator_kl_loss=1.975, generator_dur_loss=1.468, generator_adv_loss=2.228, generator_feat_match_loss=4.674, over 2001.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:41:20,640 INFO [train.py:845] (2/4) Start epoch 804
2024-02-24 08:44:51,223 INFO [train.py:845] (2/4) Start epoch 805
2024-02-24 08:45:18,323 INFO [train.py:471] (2/4) Epoch 805, batch 2, global_batch_idx: 29750, batch size: 49, loss[discriminator_loss=2.594, discriminator_real_loss=1.42, discriminator_fake_loss=1.173, generator_loss=29.79, generator_mel_loss=19.37, generator_kl_loss=1.819, generator_dur_loss=1.496, generator_adv_loss=2.545, generator_feat_match_loss=4.562, over 49.00 samples.], tot_loss[discriminator_loss=2.655, discriminator_real_loss=1.234, discriminator_fake_loss=1.42, generator_loss=29.94, generator_mel_loss=19.55, generator_kl_loss=1.968, generator_dur_loss=1.472, generator_adv_loss=2.28, generator_feat_match_loss=4.671, over 267.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:48:23,050 INFO [train.py:845] (2/4) Start epoch 806
2024-02-24 08:49:53,356 INFO [train.py:471] (2/4) Epoch 806, batch 15, global_batch_idx: 29800, batch size: 101, loss[discriminator_loss=2.619, discriminator_real_loss=1.269, discriminator_fake_loss=1.351, generator_loss=30.22, generator_mel_loss=20.04, generator_kl_loss=2.015, generator_dur_loss=1.456, generator_adv_loss=2.246, generator_feat_match_loss=4.465, over 101.00 samples.], tot_loss[discriminator_loss=2.562, discriminator_real_loss=1.289, discriminator_fake_loss=1.273, generator_loss=30.57, generator_mel_loss=20.04, generator_kl_loss=1.953, generator_dur_loss=1.468, generator_adv_loss=2.317, generator_feat_match_loss=4.794, over 1137.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 08:49:53,358 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 08:50:01,618 INFO [train.py:534] (2/4) Epoch 806, validation: discriminator_loss=2.508, discriminator_real_loss=1.351, discriminator_fake_loss=1.156, generator_loss=31.44, generator_mel_loss=20.76, generator_kl_loss=2.087, generator_dur_loss=1.473, generator_adv_loss=2.278, generator_feat_match_loss=4.843, over 100.00 samples.
2024-02-24 08:50:01,619 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 08:51:55,336 INFO [train.py:845] (2/4) Start epoch 807
2024-02-24 08:54:37,121 INFO [train.py:471] (2/4) Epoch 807, batch 28, global_batch_idx: 29850, batch size: 95, loss[discriminator_loss=2.43, discriminator_real_loss=1.229, discriminator_fake_loss=1.199, generator_loss=30.93, generator_mel_loss=19.94, generator_kl_loss=1.914, generator_dur_loss=1.465, generator_adv_loss=2.326, generator_feat_match_loss=5.285, over 95.00 samples.], tot_loss[discriminator_loss=2.545, discriminator_real_loss=1.288, discriminator_fake_loss=1.257, generator_loss=30.34, generator_mel_loss=19.92, generator_kl_loss=1.964, generator_dur_loss=1.465, generator_adv_loss=2.294, generator_feat_match_loss=4.704, over 2237.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 4.0
2024-02-24 08:55:27,441 INFO [train.py:845] (2/4) Start epoch 808
2024-02-24 08:59:02,455 INFO [train.py:845] (2/4) Start epoch 809
2024-02-24 08:59:34,167 INFO [train.py:471] (2/4) Epoch 809, batch 4, global_batch_idx: 29900, batch size: 58, loss[discriminator_loss=2.467, discriminator_real_loss=1.264, discriminator_fake_loss=1.203, generator_loss=30.09, generator_mel_loss=19.74, generator_kl_loss=1.91, generator_dur_loss=1.498, generator_adv_loss=2.359, generator_feat_match_loss=4.578, over 58.00 samples.], tot_loss[discriminator_loss=2.517, discriminator_real_loss=1.286, discriminator_fake_loss=1.231, generator_loss=29.9, generator_mel_loss=19.68, generator_kl_loss=1.922, generator_dur_loss=1.472, generator_adv_loss=2.243, generator_feat_match_loss=4.582, over 284.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 09:02:31,976 INFO [train.py:845] (2/4) Start epoch 810
2024-02-24 09:04:19,995 INFO [train.py:471] (2/4) Epoch 810, batch 17, global_batch_idx: 29950, batch size: 49, loss[discriminator_loss=2.631, discriminator_real_loss=1.223, discriminator_fake_loss=1.408, generator_loss=30.42, generator_mel_loss=20.2, generator_kl_loss=1.902, generator_dur_loss=1.48, generator_adv_loss=2.277, generator_feat_match_loss=4.562, over 49.00 samples.], tot_loss[discriminator_loss=2.587, discriminator_real_loss=1.319, discriminator_fake_loss=1.268, generator_loss=30.48, generator_mel_loss=19.95, generator_kl_loss=1.956, generator_dur_loss=1.47, generator_adv_loss=2.294, generator_feat_match_loss=4.806, over 1289.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 4.0
2024-02-24 09:06:00,996 INFO [train.py:845] (2/4) Start epoch 811
2024-02-24 09:08:49,497 INFO [train.py:471] (2/4) Epoch 811, batch 30, global_batch_idx: 30000, batch size: 101, loss[discriminator_loss=2.447, discriminator_real_loss=1.243, discriminator_fake_loss=1.204, generator_loss=31.3, generator_mel_loss=20.18, generator_kl_loss=2.012, generator_dur_loss=1.45, generator_adv_loss=2.473, generator_feat_match_loss=5.188, over 101.00 samples.], tot_loss[discriminator_loss=2.547, discriminator_real_loss=1.28, discriminator_fake_loss=1.268, generator_loss=30.24, generator_mel_loss=19.94, generator_kl_loss=1.955, generator_dur_loss=1.467, generator_adv_loss=2.218, generator_feat_match_loss=4.659, over 2258.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 09:08:49,498 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 09:08:58,140 INFO [train.py:534] (2/4) Epoch 811, validation: discriminator_loss=2.531, discriminator_real_loss=1.278, discriminator_fake_loss=1.254, generator_loss=31.02, generator_mel_loss=20.56, generator_kl_loss=2.083, generator_dur_loss=1.477, generator_adv_loss=2.036, generator_feat_match_loss=4.86, over 100.00 samples.
2024-02-24 09:08:58,141 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 09:09:34,972 INFO [train.py:845] (2/4) Start epoch 812
2024-02-24 09:13:07,908 INFO [train.py:845] (2/4) Start epoch 813
2024-02-24 09:13:51,451 INFO [train.py:471] (2/4) Epoch 813, batch 6, global_batch_idx: 30050, batch size: 53, loss[discriminator_loss=2.359, discriminator_real_loss=1.109, discriminator_fake_loss=1.249, generator_loss=30.77, generator_mel_loss=19.51, generator_kl_loss=1.864, generator_dur_loss=1.489, generator_adv_loss=2.467, generator_feat_match_loss=5.445, over 53.00 samples.], tot_loss[discriminator_loss=2.332, discriminator_real_loss=1.183, discriminator_fake_loss=1.149, generator_loss=31.23, generator_mel_loss=19.65, generator_kl_loss=1.945, generator_dur_loss=1.477, generator_adv_loss=2.529, generator_feat_match_loss=5.626, over 488.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 09:16:36,669 INFO [train.py:845] (2/4) Start epoch 814
2024-02-24 09:18:39,721 INFO [train.py:471] (2/4) Epoch 814, batch 19, global_batch_idx: 30100, batch size: 69, loss[discriminator_loss=2.504, discriminator_real_loss=1.268, discriminator_fake_loss=1.235, generator_loss=30.54, generator_mel_loss=19.85, generator_kl_loss=2.046, generator_dur_loss=1.476, generator_adv_loss=2.219, generator_feat_match_loss=4.941, over 69.00 samples.], tot_loss[discriminator_loss=2.503, discriminator_real_loss=1.261, discriminator_fake_loss=1.242, generator_loss=30.58, generator_mel_loss=19.95, generator_kl_loss=1.973, generator_dur_loss=1.472, generator_adv_loss=2.28, generator_feat_match_loss=4.901, over 1508.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 09:20:02,576 INFO [train.py:845] (2/4) Start epoch 815
2024-02-24 09:23:14,453 INFO [train.py:471] (2/4) Epoch 815, batch 32, global_batch_idx: 30150, batch size: 60, loss[discriminator_loss=2.496, discriminator_real_loss=1.308, discriminator_fake_loss=1.188, generator_loss=31.22, generator_mel_loss=19.74, generator_kl_loss=1.961, generator_dur_loss=1.464, generator_adv_loss=2.48, generator_feat_match_loss=5.566, over 60.00 samples.], tot_loss[discriminator_loss=2.545, discriminator_real_loss=1.318, discriminator_fake_loss=1.227, generator_loss=30.62, generator_mel_loss=19.78, generator_kl_loss=1.964, generator_dur_loss=1.467, generator_adv_loss=2.414, generator_feat_match_loss=4.99, over 2497.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 4.0
2024-02-24 09:23:31,414 INFO [train.py:845] (2/4) Start epoch 816
2024-02-24 09:27:03,459 INFO [train.py:845] (2/4) Start epoch 817
2024-02-24 09:28:01,946 INFO [train.py:471] (2/4) Epoch 817, batch 8, global_batch_idx: 30200, batch size: 95, loss[discriminator_loss=2.57, discriminator_real_loss=1.345, discriminator_fake_loss=1.225, generator_loss=29.96, generator_mel_loss=19.84, generator_kl_loss=1.851, generator_dur_loss=1.456, generator_adv_loss=2.178, generator_feat_match_loss=4.633, over 95.00 samples.], tot_loss[discriminator_loss=2.516, discriminator_real_loss=1.289, discriminator_fake_loss=1.227, generator_loss=30.12, generator_mel_loss=19.67, generator_kl_loss=1.966, generator_dur_loss=1.464, generator_adv_loss=2.29, generator_feat_match_loss=4.73, over 688.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 09:28:01,948 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 09:28:10,655 INFO [train.py:534] (2/4) Epoch 817, validation: discriminator_loss=2.487, discriminator_real_loss=1.243, discriminator_fake_loss=1.244, generator_loss=31.26, generator_mel_loss=20.23, generator_kl_loss=2.19, generator_dur_loss=1.469, generator_adv_loss=2.276, generator_feat_match_loss=5.096, over 100.00 samples.
2024-02-24 09:28:10,655 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 09:30:45,419 INFO [train.py:845] (2/4) Start epoch 818
2024-02-24 09:33:06,599 INFO [train.py:471] (2/4) Epoch 818, batch 21, global_batch_idx: 30250, batch size: 52, loss[discriminator_loss=2.512, discriminator_real_loss=1.088, discriminator_fake_loss=1.425, generator_loss=30.07, generator_mel_loss=19.28, generator_kl_loss=1.88, generator_dur_loss=1.478, generator_adv_loss=2.336, generator_feat_match_loss=5.094, over 52.00 samples.], tot_loss[discriminator_loss=2.402, discriminator_real_loss=1.222, discriminator_fake_loss=1.18, generator_loss=30.62, generator_mel_loss=19.53, generator_kl_loss=1.949, generator_dur_loss=1.467, generator_adv_loss=2.449, generator_feat_match_loss=5.228, over 1641.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 09:34:18,879 INFO [train.py:845] (2/4) Start epoch 819
2024-02-24 09:37:33,783 INFO [train.py:471] (2/4) Epoch 819, batch 34, global_batch_idx: 30300, batch size: 65, loss[discriminator_loss=2.582, discriminator_real_loss=1.15, discriminator_fake_loss=1.431, generator_loss=30.79, generator_mel_loss=19.99, generator_kl_loss=1.984, generator_dur_loss=1.476, generator_adv_loss=2.41, generator_feat_match_loss=4.93, over 65.00 samples.], tot_loss[discriminator_loss=2.484, discriminator_real_loss=1.259, discriminator_fake_loss=1.225, generator_loss=30.57, generator_mel_loss=19.79, generator_kl_loss=1.984, generator_dur_loss=1.467, generator_adv_loss=2.369, generator_feat_match_loss=4.964, over 2629.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 8.0
2024-02-24 09:37:48,854 INFO [train.py:845] (2/4) Start epoch 820
2024-02-24 09:41:12,499 INFO [train.py:845] (2/4) Start epoch 821
2024-02-24 09:42:18,881 INFO [train.py:471] (2/4) Epoch 821, batch 10, global_batch_idx: 30350, batch size: 69, loss[discriminator_loss=2.525, discriminator_real_loss=1.357, discriminator_fake_loss=1.168, generator_loss=30.05, generator_mel_loss=19.51, generator_kl_loss=1.934, generator_dur_loss=1.464, generator_adv_loss=2.326, generator_feat_match_loss=4.816, over 69.00 samples.], tot_loss[discriminator_loss=2.513, discriminator_real_loss=1.262, discriminator_fake_loss=1.251, generator_loss=30.32, generator_mel_loss=19.93, generator_kl_loss=1.931, generator_dur_loss=1.474, generator_adv_loss=2.29, generator_feat_match_loss=4.688, over 749.00 samples.], cur_lr_g: 1.81e-04, cur_lr_d: 1.81e-04, grad_scale: 4.0
2024-02-24 09:44:38,528 INFO [train.py:845] (2/4) Start epoch 822
2024-02-24 09:46:59,017 INFO [train.py:471] (2/4) Epoch 822, batch 23, global_batch_idx: 30400, batch size: 63, loss[discriminator_loss=2.295, discriminator_real_loss=1.266, discriminator_fake_loss=1.029, generator_loss=30.56, generator_mel_loss=18.98, generator_kl_loss=1.932, generator_dur_loss=1.487, generator_adv_loss=2.611, generator_feat_match_loss=5.547, over 63.00 samples.], tot_loss[discriminator_loss=2.501, discriminator_real_loss=1.256, discriminator_fake_loss=1.244, generator_loss=30.44, generator_mel_loss=19.53, generator_kl_loss=1.93, generator_dur_loss=1.469, generator_adv_loss=2.411, generator_feat_match_loss=5.093, over 1669.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 09:46:59,019 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 09:47:07,638 INFO [train.py:534] (2/4) Epoch 822, validation: discriminator_loss=2.225, discriminator_real_loss=1.095, discriminator_fake_loss=1.13, generator_loss=31.59, generator_mel_loss=19.82, generator_kl_loss=2.032, generator_dur_loss=1.473, generator_adv_loss=2.522, generator_feat_match_loss=5.739, over 100.00 samples.
2024-02-24 09:47:07,639 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 09:48:15,940 INFO [train.py:845] (2/4) Start epoch 823
2024-02-24 09:51:47,182 INFO [train.py:471] (2/4) Epoch 823, batch 36, global_batch_idx: 30450, batch size: 60, loss[discriminator_loss=2.504, discriminator_real_loss=1.338, discriminator_fake_loss=1.166, generator_loss=30.77, generator_mel_loss=19.67, generator_kl_loss=1.949, generator_dur_loss=1.473, generator_adv_loss=2.531, generator_feat_match_loss=5.145, over 60.00 samples.], tot_loss[discriminator_loss=2.519, discriminator_real_loss=1.278, discriminator_fake_loss=1.24, generator_loss=30.32, generator_mel_loss=19.83, generator_kl_loss=1.957, generator_dur_loss=1.468, generator_adv_loss=2.274, generator_feat_match_loss=4.792, over 2960.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 4.0
2024-02-24 09:51:47,574 INFO [train.py:845] (2/4) Start epoch 824
2024-02-24 09:55:08,951 INFO [train.py:845] (2/4) Start epoch 825
2024-02-24 09:56:30,601 INFO [train.py:471] (2/4) Epoch 825, batch 12, global_batch_idx: 30500, batch size: 81, loss[discriminator_loss=2.516, discriminator_real_loss=1.197, discriminator_fake_loss=1.317, generator_loss=31.02, generator_mel_loss=20.03, generator_kl_loss=2.06, generator_dur_loss=1.457, generator_adv_loss=2.434, generator_feat_match_loss=5.035, over 81.00 samples.], tot_loss[discriminator_loss=2.554, discriminator_real_loss=1.292, discriminator_fake_loss=1.262, generator_loss=30.55, generator_mel_loss=19.86, generator_kl_loss=1.955, generator_dur_loss=1.469, generator_adv_loss=2.351, generator_feat_match_loss=4.914, over 892.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 09:58:36,848 INFO [train.py:845] (2/4) Start epoch 826
2024-02-24 10:01:06,124 INFO [train.py:471] (2/4) Epoch 826, batch 25, global_batch_idx: 30550, batch size: 50, loss[discriminator_loss=2.824, discriminator_real_loss=1.244, discriminator_fake_loss=1.579, generator_loss=28.73, generator_mel_loss=18.94, generator_kl_loss=1.9, generator_dur_loss=1.475, generator_adv_loss=2.035, generator_feat_match_loss=4.387, over 50.00 samples.], tot_loss[discriminator_loss=2.419, discriminator_real_loss=1.204, discriminator_fake_loss=1.215, generator_loss=30.73, generator_mel_loss=19.55, generator_kl_loss=1.934, generator_dur_loss=1.47, generator_adv_loss=2.419, generator_feat_match_loss=5.359, over 1731.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:02:01,475 INFO [train.py:845] (2/4) Start epoch 827
2024-02-24 10:05:36,446 INFO [train.py:845] (2/4) Start epoch 828
2024-02-24 10:05:56,263 INFO [train.py:471] (2/4) Epoch 828, batch 1, global_batch_idx: 30600, batch size: 63, loss[discriminator_loss=2.574, discriminator_real_loss=1.226, discriminator_fake_loss=1.349, generator_loss=29.53, generator_mel_loss=19.19, generator_kl_loss=1.942, generator_dur_loss=1.493, generator_adv_loss=2.291, generator_feat_match_loss=4.605, over 63.00 samples.], tot_loss[discriminator_loss=2.508, discriminator_real_loss=1.236, discriminator_fake_loss=1.272, generator_loss=29.91, generator_mel_loss=19.4, generator_kl_loss=1.934, generator_dur_loss=1.477, generator_adv_loss=2.331, generator_feat_match_loss=4.771, over 142.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:05:56,264 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 10:06:05,468 INFO [train.py:534] (2/4) Epoch 828, validation: discriminator_loss=2.618, discriminator_real_loss=1.27, discriminator_fake_loss=1.348, generator_loss=30.23, generator_mel_loss=20.33, generator_kl_loss=2.141, generator_dur_loss=1.474, generator_adv_loss=1.9, generator_feat_match_loss=4.378, over 100.00 samples.
2024-02-24 10:06:05,468 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 10:09:12,729 INFO [train.py:845] (2/4) Start epoch 829
2024-02-24 10:10:44,253 INFO [train.py:471] (2/4) Epoch 829, batch 14, global_batch_idx: 30650, batch size: 90, loss[discriminator_loss=2.688, discriminator_real_loss=1.336, discriminator_fake_loss=1.352, generator_loss=30.62, generator_mel_loss=20.11, generator_kl_loss=1.984, generator_dur_loss=1.454, generator_adv_loss=2.223, generator_feat_match_loss=4.844, over 90.00 samples.], tot_loss[discriminator_loss=2.462, discriminator_real_loss=1.217, discriminator_fake_loss=1.246, generator_loss=30.65, generator_mel_loss=19.62, generator_kl_loss=1.942, generator_dur_loss=1.467, generator_adv_loss=2.402, generator_feat_match_loss=5.224, over 1126.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:12:43,039 INFO [train.py:845] (2/4) Start epoch 830
2024-02-24 10:15:20,143 INFO [train.py:471] (2/4) Epoch 830, batch 27, global_batch_idx: 30700, batch size: 79, loss[discriminator_loss=2.4, discriminator_real_loss=1.271, discriminator_fake_loss=1.129, generator_loss=30.68, generator_mel_loss=19.62, generator_kl_loss=2.012, generator_dur_loss=1.475, generator_adv_loss=2.391, generator_feat_match_loss=5.176, over 79.00 samples.], tot_loss[discriminator_loss=2.447, discriminator_real_loss=1.229, discriminator_fake_loss=1.219, generator_loss=30.55, generator_mel_loss=19.65, generator_kl_loss=1.968, generator_dur_loss=1.473, generator_adv_loss=2.395, generator_feat_match_loss=5.065, over 1939.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:16:13,017 INFO [train.py:845] (2/4) Start epoch 831
2024-02-24 10:19:46,443 INFO [train.py:845] (2/4) Start epoch 832
2024-02-24 10:20:17,848 INFO [train.py:471] (2/4) Epoch 832, batch 3, global_batch_idx: 30750, batch size: 81, loss[discriminator_loss=2.359, discriminator_real_loss=1.223, discriminator_fake_loss=1.136, generator_loss=30.62, generator_mel_loss=19.29, generator_kl_loss=1.984, generator_dur_loss=1.47, generator_adv_loss=2.545, generator_feat_match_loss=5.336, over 81.00 samples.], tot_loss[discriminator_loss=2.416, discriminator_real_loss=1.224, discriminator_fake_loss=1.192, generator_loss=30.57, generator_mel_loss=19.52, generator_kl_loss=1.937, generator_dur_loss=1.466, generator_adv_loss=2.432, generator_feat_match_loss=5.212, over 346.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 4.0
2024-02-24 10:23:11,711 INFO [train.py:845] (2/4) Start epoch 833
2024-02-24 10:24:47,378 INFO [train.py:471] (2/4) Epoch 833, batch 16, global_batch_idx: 30800, batch size: 110, loss[discriminator_loss=2.426, discriminator_real_loss=1.234, discriminator_fake_loss=1.192, generator_loss=30.54, generator_mel_loss=19.87, generator_kl_loss=1.941, generator_dur_loss=1.462, generator_adv_loss=2.396, generator_feat_match_loss=4.867, over 110.00 samples.], tot_loss[discriminator_loss=2.474, discriminator_real_loss=1.24, discriminator_fake_loss=1.234, generator_loss=30.33, generator_mel_loss=19.55, generator_kl_loss=1.943, generator_dur_loss=1.468, generator_adv_loss=2.354, generator_feat_match_loss=5.009, over 1228.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:24:47,380 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 10:24:56,352 INFO [train.py:534] (2/4) Epoch 833, validation: discriminator_loss=2.413, discriminator_real_loss=1.246, discriminator_fake_loss=1.168, generator_loss=31.07, generator_mel_loss=20.27, generator_kl_loss=2.041, generator_dur_loss=1.467, generator_adv_loss=2.25, generator_feat_match_loss=5.05, over 100.00 samples.
2024-02-24 10:24:56,353 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 10:26:45,208 INFO [train.py:845] (2/4) Start epoch 834
2024-02-24 10:29:26,783 INFO [train.py:471] (2/4) Epoch 834, batch 29, global_batch_idx: 30850, batch size: 63, loss[discriminator_loss=2.438, discriminator_real_loss=1.176, discriminator_fake_loss=1.261, generator_loss=30.4, generator_mel_loss=19.34, generator_kl_loss=1.896, generator_dur_loss=1.493, generator_adv_loss=2.4, generator_feat_match_loss=5.266, over 63.00 samples.], tot_loss[discriminator_loss=2.442, discriminator_real_loss=1.24, discriminator_fake_loss=1.202, generator_loss=30.85, generator_mel_loss=19.78, generator_kl_loss=1.953, generator_dur_loss=1.471, generator_adv_loss=2.411, generator_feat_match_loss=5.227, over 2075.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 4.0
2024-02-24 10:30:06,138 INFO [train.py:845] (2/4) Start epoch 835
2024-02-24 10:33:33,613 INFO [train.py:845] (2/4) Start epoch 836
2024-02-24 10:34:07,445 INFO [train.py:471] (2/4) Epoch 836, batch 5, global_batch_idx: 30900, batch size: 67, loss[discriminator_loss=2.922, discriminator_real_loss=1.242, discriminator_fake_loss=1.68, generator_loss=28.81, generator_mel_loss=19.35, generator_kl_loss=1.942, generator_dur_loss=1.47, generator_adv_loss=1.947, generator_feat_match_loss=4.102, over 67.00 samples.], tot_loss[discriminator_loss=2.397, discriminator_real_loss=1.15, discriminator_fake_loss=1.247, generator_loss=30.61, generator_mel_loss=19.43, generator_kl_loss=1.959, generator_dur_loss=1.471, generator_adv_loss=2.394, generator_feat_match_loss=5.349, over 384.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:36:55,200 INFO [train.py:845] (2/4) Start epoch 837
2024-02-24 10:38:53,049 INFO [train.py:471] (2/4) Epoch 837, batch 18, global_batch_idx: 30950, batch size: 63, loss[discriminator_loss=2.371, discriminator_real_loss=1.184, discriminator_fake_loss=1.188, generator_loss=30.79, generator_mel_loss=19.29, generator_kl_loss=1.861, generator_dur_loss=1.464, generator_adv_loss=2.543, generator_feat_match_loss=5.633, over 63.00 samples.], tot_loss[discriminator_loss=2.423, discriminator_real_loss=1.228, discriminator_fake_loss=1.195, generator_loss=30.74, generator_mel_loss=19.44, generator_kl_loss=1.964, generator_dur_loss=1.463, generator_adv_loss=2.485, generator_feat_match_loss=5.393, over 1472.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:40:27,527 INFO [train.py:845] (2/4) Start epoch 838
2024-02-24 10:43:30,534 INFO [train.py:471] (2/4) Epoch 838, batch 31, global_batch_idx: 31000, batch size: 58, loss[discriminator_loss=2.258, discriminator_real_loss=1.136, discriminator_fake_loss=1.123, generator_loss=30.96, generator_mel_loss=19.25, generator_kl_loss=1.949, generator_dur_loss=1.491, generator_adv_loss=2.541, generator_feat_match_loss=5.723, over 58.00 samples.], tot_loss[discriminator_loss=2.376, discriminator_real_loss=1.2, discriminator_fake_loss=1.176, generator_loss=30.79, generator_mel_loss=19.4, generator_kl_loss=1.947, generator_dur_loss=1.465, generator_adv_loss=2.506, generator_feat_match_loss=5.473, over 2479.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:43:30,536 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 10:43:38,976 INFO [train.py:534] (2/4) Epoch 838, validation: discriminator_loss=2.224, discriminator_real_loss=1.015, discriminator_fake_loss=1.209, generator_loss=31.96, generator_mel_loss=19.92, generator_kl_loss=2.163, generator_dur_loss=1.475, generator_adv_loss=2.5, generator_feat_match_loss=5.904, over 100.00 samples.
2024-02-24 10:43:38,977 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 10:44:05,861 INFO [train.py:845] (2/4) Start epoch 839
2024-02-24 10:47:33,459 INFO [train.py:845] (2/4) Start epoch 840
2024-02-24 10:48:31,686 INFO [train.py:471] (2/4) Epoch 840, batch 7, global_batch_idx: 31050, batch size: 59, loss[discriminator_loss=2.688, discriminator_real_loss=1.595, discriminator_fake_loss=1.094, generator_loss=31.27, generator_mel_loss=20.23, generator_kl_loss=1.969, generator_dur_loss=1.496, generator_adv_loss=2.494, generator_feat_match_loss=5.086, over 59.00 samples.], tot_loss[discriminator_loss=2.533, discriminator_real_loss=1.316, discriminator_fake_loss=1.217, generator_loss=30.78, generator_mel_loss=19.87, generator_kl_loss=1.984, generator_dur_loss=1.46, generator_adv_loss=2.483, generator_feat_match_loss=4.989, over 625.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 4.0
2024-02-24 10:51:04,836 INFO [train.py:845] (2/4) Start epoch 841
2024-02-24 10:52:54,793 INFO [train.py:471] (2/4) Epoch 841, batch 20, global_batch_idx: 31100, batch size: 95, loss[discriminator_loss=2.377, discriminator_real_loss=1.196, discriminator_fake_loss=1.181, generator_loss=30.84, generator_mel_loss=19.69, generator_kl_loss=1.959, generator_dur_loss=1.443, generator_adv_loss=2.506, generator_feat_match_loss=5.246, over 95.00 samples.], tot_loss[discriminator_loss=2.364, discriminator_real_loss=1.192, discriminator_fake_loss=1.172, generator_loss=30.85, generator_mel_loss=19.42, generator_kl_loss=1.924, generator_dur_loss=1.472, generator_adv_loss=2.506, generator_feat_match_loss=5.529, over 1377.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:54:24,726 INFO [train.py:845] (2/4) Start epoch 842
2024-02-24 10:57:26,623 INFO [train.py:471] (2/4) Epoch 842, batch 33, global_batch_idx: 31150, batch size: 61, loss[discriminator_loss=2.334, discriminator_real_loss=1.161, discriminator_fake_loss=1.173, generator_loss=30.55, generator_mel_loss=19.14, generator_kl_loss=2.024, generator_dur_loss=1.496, generator_adv_loss=2.438, generator_feat_match_loss=5.453, over 61.00 samples.], tot_loss[discriminator_loss=2.347, discriminator_real_loss=1.195, discriminator_fake_loss=1.152, generator_loss=30.97, generator_mel_loss=19.41, generator_kl_loss=1.953, generator_dur_loss=1.468, generator_adv_loss=2.534, generator_feat_match_loss=5.602, over 2609.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 10:57:46,026 INFO [train.py:845] (2/4) Start epoch 843
2024-02-24 11:01:13,554 INFO [train.py:845] (2/4) Start epoch 844
2024-02-24 11:02:10,140 INFO [train.py:471] (2/4) Epoch 844, batch 9, global_batch_idx: 31200, batch size: 64, loss[discriminator_loss=2.35, discriminator_real_loss=1.298, discriminator_fake_loss=1.052, generator_loss=30.72, generator_mel_loss=19.23, generator_kl_loss=1.998, generator_dur_loss=1.465, generator_adv_loss=2.578, generator_feat_match_loss=5.449, over 64.00 samples.], tot_loss[discriminator_loss=2.434, discriminator_real_loss=1.234, discriminator_fake_loss=1.201, generator_loss=30.45, generator_mel_loss=19.24, generator_kl_loss=1.962, generator_dur_loss=1.466, generator_adv_loss=2.465, generator_feat_match_loss=5.315, over 689.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 16.0
2024-02-24 11:02:10,142 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 11:02:19,381 INFO [train.py:534] (2/4) Epoch 844, validation: discriminator_loss=2.289, discriminator_real_loss=1.157, discriminator_fake_loss=1.132, generator_loss=31.4, generator_mel_loss=19.84, generator_kl_loss=2.202, generator_dur_loss=1.473, generator_adv_loss=2.399, generator_feat_match_loss=5.487, over 100.00 samples.
2024-02-24 11:02:19,382 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 11:04:49,183 INFO [train.py:845] (2/4) Start epoch 845
2024-02-24 11:07:01,219 INFO [train.py:471] (2/4) Epoch 845, batch 22, global_batch_idx: 31250, batch size: 65, loss[discriminator_loss=2.461, discriminator_real_loss=1.145, discriminator_fake_loss=1.317, generator_loss=30.63, generator_mel_loss=19.85, generator_kl_loss=1.882, generator_dur_loss=1.48, generator_adv_loss=2.449, generator_feat_match_loss=4.965, over 65.00 samples.], tot_loss[discriminator_loss=2.508, discriminator_real_loss=1.264, discriminator_fake_loss=1.244, generator_loss=30.5, generator_mel_loss=19.71, generator_kl_loss=1.967, generator_dur_loss=1.468, generator_adv_loss=2.34, generator_feat_match_loss=5.015, over 1626.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:08:19,849 INFO [train.py:845] (2/4) Start epoch 846
2024-02-24 11:11:44,525 INFO [train.py:471] (2/4) Epoch 846, batch 35, global_batch_idx: 31300, batch size: 54, loss[discriminator_loss=2.426, discriminator_real_loss=1.3, discriminator_fake_loss=1.125, generator_loss=31.39, generator_mel_loss=20.22, generator_kl_loss=1.989, generator_dur_loss=1.477, generator_adv_loss=2.332, generator_feat_match_loss=5.371, over 54.00 samples.], tot_loss[discriminator_loss=2.47, discriminator_real_loss=1.243, discriminator_fake_loss=1.226, generator_loss=30.58, generator_mel_loss=19.77, generator_kl_loss=1.958, generator_dur_loss=1.466, generator_adv_loss=2.338, generator_feat_match_loss=5.05, over 2778.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:11:49,419 INFO [train.py:845] (2/4) Start epoch 847
2024-02-24 11:15:23,681 INFO [train.py:845] (2/4) Start epoch 848
2024-02-24 11:16:31,432 INFO [train.py:471] (2/4) Epoch 848, batch 11, global_batch_idx: 31350, batch size: 63, loss[discriminator_loss=2.301, discriminator_real_loss=1.093, discriminator_fake_loss=1.207, generator_loss=32.01, generator_mel_loss=19.8, generator_kl_loss=1.922, generator_dur_loss=1.483, generator_adv_loss=2.727, generator_feat_match_loss=6.078, over 63.00 samples.], tot_loss[discriminator_loss=2.408, discriminator_real_loss=1.205, discriminator_fake_loss=1.203, generator_loss=30.77, generator_mel_loss=19.33, generator_kl_loss=1.951, generator_dur_loss=1.471, generator_adv_loss=2.531, generator_feat_match_loss=5.478, over 789.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:18:52,088 INFO [train.py:845] (2/4) Start epoch 849
2024-02-24 11:21:17,411 INFO [train.py:471] (2/4) Epoch 849, batch 24, global_batch_idx: 31400, batch size: 53, loss[discriminator_loss=2.254, discriminator_real_loss=1.158, discriminator_fake_loss=1.096, generator_loss=30.66, generator_mel_loss=19.03, generator_kl_loss=2.003, generator_dur_loss=1.494, generator_adv_loss=2.473, generator_feat_match_loss=5.668, over 53.00 samples.], tot_loss[discriminator_loss=2.459, discriminator_real_loss=1.253, discriminator_fake_loss=1.206, generator_loss=30.59, generator_mel_loss=19.51, generator_kl_loss=1.965, generator_dur_loss=1.474, generator_adv_loss=2.43, generator_feat_match_loss=5.208, over 1622.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:21:17,413 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 11:21:25,996 INFO [train.py:534] (2/4) Epoch 849, validation: discriminator_loss=2.216, discriminator_real_loss=1.036, discriminator_fake_loss=1.18, generator_loss=32.39, generator_mel_loss=20.15, generator_kl_loss=2.091, generator_dur_loss=1.472, generator_adv_loss=2.431, generator_feat_match_loss=6.246, over 100.00 samples.
2024-02-24 11:21:25,997 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 11:22:32,189 INFO [train.py:845] (2/4) Start epoch 850
2024-02-24 11:25:57,254 INFO [train.py:845] (2/4) Start epoch 851
2024-02-24 11:26:10,186 INFO [train.py:471] (2/4) Epoch 851, batch 0, global_batch_idx: 31450, batch size: 52, loss[discriminator_loss=2.58, discriminator_real_loss=1.186, discriminator_fake_loss=1.395, generator_loss=30.19, generator_mel_loss=19.68, generator_kl_loss=1.902, generator_dur_loss=1.462, generator_adv_loss=2.082, generator_feat_match_loss=5.07, over 52.00 samples.], tot_loss[discriminator_loss=2.58, discriminator_real_loss=1.186, discriminator_fake_loss=1.395, generator_loss=30.19, generator_mel_loss=19.68, generator_kl_loss=1.902, generator_dur_loss=1.462, generator_adv_loss=2.082, generator_feat_match_loss=5.07, over 52.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:29:30,160 INFO [train.py:845] (2/4) Start epoch 852
2024-02-24 11:30:57,702 INFO [train.py:471] (2/4) Epoch 852, batch 13, global_batch_idx: 31500, batch size: 49, loss[discriminator_loss=2.797, discriminator_real_loss=1.202, discriminator_fake_loss=1.596, generator_loss=29.17, generator_mel_loss=18.9, generator_kl_loss=1.881, generator_dur_loss=1.477, generator_adv_loss=2.24, generator_feat_match_loss=4.668, over 49.00 samples.], tot_loss[discriminator_loss=2.358, discriminator_real_loss=1.188, discriminator_fake_loss=1.17, generator_loss=30.96, generator_mel_loss=19.44, generator_kl_loss=1.947, generator_dur_loss=1.47, generator_adv_loss=2.534, generator_feat_match_loss=5.566, over 915.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:32:56,339 INFO [train.py:845] (2/4) Start epoch 853
2024-02-24 11:35:34,905 INFO [train.py:471] (2/4) Epoch 853, batch 26, global_batch_idx: 31550, batch size: 101, loss[discriminator_loss=2.445, discriminator_real_loss=1.372, discriminator_fake_loss=1.073, generator_loss=31.04, generator_mel_loss=19.85, generator_kl_loss=1.947, generator_dur_loss=1.463, generator_adv_loss=2.514, generator_feat_match_loss=5.262, over 101.00 samples.], tot_loss[discriminator_loss=2.397, discriminator_real_loss=1.207, discriminator_fake_loss=1.19, generator_loss=30.56, generator_mel_loss=19.32, generator_kl_loss=1.955, generator_dur_loss=1.464, generator_adv_loss=2.472, generator_feat_match_loss=5.347, over 1953.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:36:29,939 INFO [train.py:845] (2/4) Start epoch 854
2024-02-24 11:40:01,599 INFO [train.py:845] (2/4) Start epoch 855
2024-02-24 11:40:21,525 INFO [train.py:471] (2/4) Epoch 855, batch 2, global_batch_idx: 31600, batch size: 59, loss[discriminator_loss=2.244, discriminator_real_loss=1.148, discriminator_fake_loss=1.096, generator_loss=30.69, generator_mel_loss=19.17, generator_kl_loss=1.892, generator_dur_loss=1.483, generator_adv_loss=2.49, generator_feat_match_loss=5.652, over 59.00 samples.], tot_loss[discriminator_loss=2.282, discriminator_real_loss=1.158, discriminator_fake_loss=1.124, generator_loss=30.69, generator_mel_loss=19.13, generator_kl_loss=1.905, generator_dur_loss=1.474, generator_adv_loss=2.543, generator_feat_match_loss=5.644, over 193.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:40:21,527 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 11:40:30,296 INFO [train.py:534] (2/4) Epoch 855, validation: discriminator_loss=2.228, discriminator_real_loss=1.059, discriminator_fake_loss=1.169, generator_loss=31.7, generator_mel_loss=19.85, generator_kl_loss=2.182, generator_dur_loss=1.474, generator_adv_loss=2.45, generator_feat_match_loss=5.742, over 100.00 samples.
2024-02-24 11:40:30,297 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 11:43:38,898 INFO [train.py:845] (2/4) Start epoch 856
2024-02-24 11:45:10,731 INFO [train.py:471] (2/4) Epoch 856, batch 15, global_batch_idx: 31650, batch size: 64, loss[discriminator_loss=2.518, discriminator_real_loss=1.118, discriminator_fake_loss=1.399, generator_loss=29.65, generator_mel_loss=18.96, generator_kl_loss=1.899, generator_dur_loss=1.472, generator_adv_loss=2.297, generator_feat_match_loss=5.016, over 64.00 samples.], tot_loss[discriminator_loss=2.373, discriminator_real_loss=1.178, discriminator_fake_loss=1.196, generator_loss=30.61, generator_mel_loss=19.29, generator_kl_loss=1.938, generator_dur_loss=1.47, generator_adv_loss=2.477, generator_feat_match_loss=5.439, over 1059.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:47:09,159 INFO [train.py:845] (2/4) Start epoch 857
2024-02-24 11:49:48,312 INFO [train.py:471] (2/4) Epoch 857, batch 28, global_batch_idx: 31700, batch size: 63, loss[discriminator_loss=2.543, discriminator_real_loss=1.219, discriminator_fake_loss=1.324, generator_loss=30.37, generator_mel_loss=19.93, generator_kl_loss=1.906, generator_dur_loss=1.493, generator_adv_loss=2.115, generator_feat_match_loss=4.93, over 63.00 samples.], tot_loss[discriminator_loss=2.519, discriminator_real_loss=1.268, discriminator_fake_loss=1.25, generator_loss=30.6, generator_mel_loss=19.91, generator_kl_loss=1.965, generator_dur_loss=1.471, generator_adv_loss=2.328, generator_feat_match_loss=4.927, over 2082.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:50:34,426 INFO [train.py:845] (2/4) Start epoch 858
2024-02-24 11:54:05,675 INFO [train.py:845] (2/4) Start epoch 859
2024-02-24 11:54:40,042 INFO [train.py:471] (2/4) Epoch 859, batch 4, global_batch_idx: 31750, batch size: 51, loss[discriminator_loss=2.203, discriminator_real_loss=1.114, discriminator_fake_loss=1.088, generator_loss=31.81, generator_mel_loss=19.49, generator_kl_loss=2.04, generator_dur_loss=1.46, generator_adv_loss=2.756, generator_feat_match_loss=6.066, over 51.00 samples.], tot_loss[discriminator_loss=2.291, discriminator_real_loss=1.149, discriminator_fake_loss=1.142, generator_loss=31.11, generator_mel_loss=19.37, generator_kl_loss=1.977, generator_dur_loss=1.473, generator_adv_loss=2.594, generator_feat_match_loss=5.701, over 343.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:57:39,265 INFO [train.py:845] (2/4) Start epoch 860
2024-02-24 11:59:25,733 INFO [train.py:471] (2/4) Epoch 860, batch 17, global_batch_idx: 31800, batch size: 95, loss[discriminator_loss=2.453, discriminator_real_loss=1.306, discriminator_fake_loss=1.146, generator_loss=31.44, generator_mel_loss=19.79, generator_kl_loss=1.998, generator_dur_loss=1.455, generator_adv_loss=2.682, generator_feat_match_loss=5.523, over 95.00 samples.], tot_loss[discriminator_loss=2.381, discriminator_real_loss=1.19, discriminator_fake_loss=1.191, generator_loss=30.58, generator_mel_loss=19.25, generator_kl_loss=1.966, generator_dur_loss=1.467, generator_adv_loss=2.49, generator_feat_match_loss=5.41, over 1235.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 11:59:25,735 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 11:59:34,162 INFO [train.py:534] (2/4) Epoch 860, validation: discriminator_loss=2.437, discriminator_real_loss=1.299, discriminator_fake_loss=1.137, generator_loss=31.8, generator_mel_loss=20.22, generator_kl_loss=2.105, generator_dur_loss=1.469, generator_adv_loss=2.549, generator_feat_match_loss=5.452, over 100.00 samples.
2024-02-24 11:59:34,163 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 12:01:21,489 INFO [train.py:845] (2/4) Start epoch 861
2024-02-24 12:04:13,450 INFO [train.py:471] (2/4) Epoch 861, batch 30, global_batch_idx: 31850, batch size: 51, loss[discriminator_loss=2.465, discriminator_real_loss=1.32, discriminator_fake_loss=1.146, generator_loss=30.35, generator_mel_loss=19.5, generator_kl_loss=2.057, generator_dur_loss=1.484, generator_adv_loss=2.4, generator_feat_match_loss=4.906, over 51.00 samples.], tot_loss[discriminator_loss=2.407, discriminator_real_loss=1.216, discriminator_fake_loss=1.191, generator_loss=30.63, generator_mel_loss=19.52, generator_kl_loss=1.97, generator_dur_loss=1.471, generator_adv_loss=2.407, generator_feat_match_loss=5.261, over 2165.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 4.0
2024-02-24 12:04:52,855 INFO [train.py:845] (2/4) Start epoch 862
2024-02-24 12:08:20,377 INFO [train.py:845] (2/4) Start epoch 863
2024-02-24 12:09:04,568 INFO [train.py:471] (2/4) Epoch 863, batch 6, global_batch_idx: 31900, batch size: 85, loss[discriminator_loss=2.428, discriminator_real_loss=1.229, discriminator_fake_loss=1.199, generator_loss=30.8, generator_mel_loss=19.44, generator_kl_loss=1.983, generator_dur_loss=1.459, generator_adv_loss=2.568, generator_feat_match_loss=5.348, over 85.00 samples.], tot_loss[discriminator_loss=2.295, discriminator_real_loss=1.131, discriminator_fake_loss=1.163, generator_loss=31.39, generator_mel_loss=19.38, generator_kl_loss=1.988, generator_dur_loss=1.476, generator_adv_loss=2.707, generator_feat_match_loss=5.837, over 517.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 12:11:50,350 INFO [train.py:845] (2/4) Start epoch 864
2024-02-24 12:13:53,677 INFO [train.py:471] (2/4) Epoch 864, batch 19, global_batch_idx: 31950, batch size: 110, loss[discriminator_loss=2.539, discriminator_real_loss=1.342, discriminator_fake_loss=1.196, generator_loss=30.36, generator_mel_loss=19.59, generator_kl_loss=1.978, generator_dur_loss=1.435, generator_adv_loss=2.406, generator_feat_match_loss=4.953, over 110.00 samples.], tot_loss[discriminator_loss=2.475, discriminator_real_loss=1.272, discriminator_fake_loss=1.203, generator_loss=30.24, generator_mel_loss=19.36, generator_kl_loss=1.973, generator_dur_loss=1.476, generator_adv_loss=2.415, generator_feat_match_loss=5.017, over 1450.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 8.0
2024-02-24 12:15:20,402 INFO [train.py:845] (2/4) Start epoch 865
2024-02-24 12:18:28,315 INFO [train.py:471] (2/4) Epoch 865, batch 32, global_batch_idx: 32000, batch size: 53, loss[discriminator_loss=2.488, discriminator_real_loss=1.181, discriminator_fake_loss=1.307, generator_loss=30.28, generator_mel_loss=19.59, generator_kl_loss=1.94, generator_dur_loss=1.516, generator_adv_loss=2.406, generator_feat_match_loss=4.82, over 53.00 samples.], tot_loss[discriminator_loss=2.508, discriminator_real_loss=1.264, discriminator_fake_loss=1.243, generator_loss=30.36, generator_mel_loss=19.62, generator_kl_loss=1.959, generator_dur_loss=1.469, generator_adv_loss=2.344, generator_feat_match_loss=4.958, over 2368.00 samples.], cur_lr_g: 1.80e-04, cur_lr_d: 1.80e-04, grad_scale: 16.0
2024-02-24 12:18:28,317 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 12:18:37,267 INFO [train.py:534] (2/4) Epoch 865, validation: discriminator_loss=2.493, discriminator_real_loss=1.298, discriminator_fake_loss=1.196, generator_loss=31.81, generator_mel_loss=20.49, generator_kl_loss=2.266, generator_dur_loss=1.468, generator_adv_loss=2.409, generator_feat_match_loss=5.172, over 100.00 samples.
2024-02-24 12:18:37,268 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 12:18:57,637 INFO [train.py:845] (2/4) Start epoch 866
2024-02-24 12:22:29,446 INFO [train.py:845] (2/4) Start epoch 867
2024-02-24 12:23:30,334 INFO [train.py:471] (2/4) Epoch 867, batch 8, global_batch_idx: 32050, batch size: 76, loss[discriminator_loss=2.43, discriminator_real_loss=1.295, discriminator_fake_loss=1.135, generator_loss=30.56, generator_mel_loss=19.66, generator_kl_loss=1.849, generator_dur_loss=1.478, generator_adv_loss=2.514, generator_feat_match_loss=5.062, over 76.00 samples.], tot_loss[discriminator_loss=2.405, discriminator_real_loss=1.235, discriminator_fake_loss=1.17, generator_loss=30.96, generator_mel_loss=19.69, generator_kl_loss=1.963, generator_dur_loss=1.478, generator_adv_loss=2.478, generator_feat_match_loss=5.348, over 614.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 12:26:03,686 INFO [train.py:845] (2/4) Start epoch 868
2024-02-24 12:28:04,731 INFO [train.py:471] (2/4) Epoch 868, batch 21, global_batch_idx: 32100, batch size: 51, loss[discriminator_loss=2.387, discriminator_real_loss=1.255, discriminator_fake_loss=1.133, generator_loss=30.96, generator_mel_loss=19.58, generator_kl_loss=1.941, generator_dur_loss=1.513, generator_adv_loss=2.504, generator_feat_match_loss=5.418, over 51.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.212, discriminator_fake_loss=1.189, generator_loss=30.65, generator_mel_loss=19.45, generator_kl_loss=1.985, generator_dur_loss=1.469, generator_adv_loss=2.454, generator_feat_match_loss=5.292, over 1464.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 12:29:31,132 INFO [train.py:845] (2/4) Start epoch 869
2024-02-24 12:32:52,021 INFO [train.py:471] (2/4) Epoch 869, batch 34, global_batch_idx: 32150, batch size: 53, loss[discriminator_loss=2.307, discriminator_real_loss=1.15, discriminator_fake_loss=1.156, generator_loss=31.24, generator_mel_loss=19.45, generator_kl_loss=1.919, generator_dur_loss=1.487, generator_adv_loss=2.627, generator_feat_match_loss=5.758, over 53.00 samples.], tot_loss[discriminator_loss=2.343, discriminator_real_loss=1.181, discriminator_fake_loss=1.162, generator_loss=30.91, generator_mel_loss=19.27, generator_kl_loss=1.948, generator_dur_loss=1.468, generator_adv_loss=2.54, generator_feat_match_loss=5.68, over 2581.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 12:33:01,409 INFO [train.py:845] (2/4) Start epoch 870
2024-02-24 12:36:31,498 INFO [train.py:845] (2/4) Start epoch 871
2024-02-24 12:37:36,285 INFO [train.py:471] (2/4) Epoch 871, batch 10, global_batch_idx: 32200, batch size: 101, loss[discriminator_loss=2.291, discriminator_real_loss=1.202, discriminator_fake_loss=1.089, generator_loss=31.37, generator_mel_loss=19.32, generator_kl_loss=1.926, generator_dur_loss=1.454, generator_adv_loss=2.658, generator_feat_match_loss=6.008, over 101.00 samples.], tot_loss[discriminator_loss=2.367, discriminator_real_loss=1.2, discriminator_fake_loss=1.167, generator_loss=30.86, generator_mel_loss=19.21, generator_kl_loss=1.965, generator_dur_loss=1.469, generator_adv_loss=2.567, generator_feat_match_loss=5.654, over 805.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 12:37:36,286 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 12:37:44,145 INFO [train.py:534] (2/4) Epoch 871, validation: discriminator_loss=2.271, discriminator_real_loss=1.128, discriminator_fake_loss=1.143, generator_loss=32.55, generator_mel_loss=20.28, generator_kl_loss=2.134, generator_dur_loss=1.47, generator_adv_loss=2.526, generator_feat_match_loss=6.131, over 100.00 samples.
2024-02-24 12:37:44,145 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 12:40:05,963 INFO [train.py:845] (2/4) Start epoch 872
2024-02-24 12:42:19,810 INFO [train.py:471] (2/4) Epoch 872, batch 23, global_batch_idx: 32250, batch size: 60, loss[discriminator_loss=2.369, discriminator_real_loss=1.236, discriminator_fake_loss=1.133, generator_loss=30.12, generator_mel_loss=19.22, generator_kl_loss=1.992, generator_dur_loss=1.468, generator_adv_loss=2.439, generator_feat_match_loss=5, over 60.00 samples.], tot_loss[discriminator_loss=2.376, discriminator_real_loss=1.193, discriminator_fake_loss=1.184, generator_loss=30.67, generator_mel_loss=19.23, generator_kl_loss=1.981, generator_dur_loss=1.472, generator_adv_loss=2.501, generator_feat_match_loss=5.48, over 1556.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 12:43:28,658 INFO [train.py:845] (2/4) Start epoch 873
2024-02-24 12:46:55,277 INFO [train.py:471] (2/4) Epoch 873, batch 36, global_batch_idx: 32300, batch size: 73, loss[discriminator_loss=2.504, discriminator_real_loss=1.176, discriminator_fake_loss=1.328, generator_loss=31.29, generator_mel_loss=19.72, generator_kl_loss=1.963, generator_dur_loss=1.48, generator_adv_loss=2.436, generator_feat_match_loss=5.695, over 73.00 samples.], tot_loss[discriminator_loss=2.41, discriminator_real_loss=1.225, discriminator_fake_loss=1.185, generator_loss=30.74, generator_mel_loss=19.42, generator_kl_loss=1.957, generator_dur_loss=1.467, generator_adv_loss=2.484, generator_feat_match_loss=5.412, over 2650.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 12:46:55,727 INFO [train.py:845] (2/4) Start epoch 874
2024-02-24 12:50:29,930 INFO [train.py:845] (2/4) Start epoch 875
2024-02-24 12:51:56,398 INFO [train.py:471] (2/4) Epoch 875, batch 12, global_batch_idx: 32350, batch size: 76, loss[discriminator_loss=2.406, discriminator_real_loss=1.202, discriminator_fake_loss=1.203, generator_loss=30.04, generator_mel_loss=18.97, generator_kl_loss=1.943, generator_dur_loss=1.456, generator_adv_loss=2.449, generator_feat_match_loss=5.219, over 76.00 samples.], tot_loss[discriminator_loss=2.312, discriminator_real_loss=1.158, discriminator_fake_loss=1.154, generator_loss=30.78, generator_mel_loss=19.17, generator_kl_loss=1.939, generator_dur_loss=1.452, generator_adv_loss=2.532, generator_feat_match_loss=5.689, over 1067.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 12:53:53,756 INFO [train.py:845] (2/4) Start epoch 876
2024-02-24 12:56:19,052 INFO [train.py:471] (2/4) Epoch 876, batch 25, global_batch_idx: 32400, batch size: 53, loss[discriminator_loss=2.27, discriminator_real_loss=1.14, discriminator_fake_loss=1.13, generator_loss=30.76, generator_mel_loss=18.71, generator_kl_loss=2.016, generator_dur_loss=1.474, generator_adv_loss=2.658, generator_feat_match_loss=5.902, over 53.00 samples.], tot_loss[discriminator_loss=2.395, discriminator_real_loss=1.222, discriminator_fake_loss=1.172, generator_loss=30.84, generator_mel_loss=19.25, generator_kl_loss=1.973, generator_dur_loss=1.467, generator_adv_loss=2.554, generator_feat_match_loss=5.601, over 1844.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 16.0
2024-02-24 12:56:19,055 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 12:56:27,645 INFO [train.py:534] (2/4) Epoch 876, validation: discriminator_loss=2.212, discriminator_real_loss=1.075, discriminator_fake_loss=1.137, generator_loss=31.86, generator_mel_loss=19.68, generator_kl_loss=2.09, generator_dur_loss=1.467, generator_adv_loss=2.54, generator_feat_match_loss=6.082, over 100.00 samples.
2024-02-24 12:56:27,647 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 12:57:31,935 INFO [train.py:845] (2/4) Start epoch 877
2024-02-24 13:01:02,927 INFO [train.py:845] (2/4) Start epoch 878
2024-02-24 13:01:20,798 INFO [train.py:471] (2/4) Epoch 878, batch 1, global_batch_idx: 32450, batch size: 51, loss[discriminator_loss=2.268, discriminator_real_loss=1.132, discriminator_fake_loss=1.136, generator_loss=30.89, generator_mel_loss=19.06, generator_kl_loss=1.981, generator_dur_loss=1.481, generator_adv_loss=2.59, generator_feat_match_loss=5.781, over 51.00 samples.], tot_loss[discriminator_loss=2.288, discriminator_real_loss=1.164, discriminator_fake_loss=1.124, generator_loss=30.75, generator_mel_loss=19.09, generator_kl_loss=1.996, generator_dur_loss=1.467, generator_adv_loss=2.558, generator_feat_match_loss=5.637, over 102.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 16.0
2024-02-24 13:04:35,585 INFO [train.py:845] (2/4) Start epoch 879
2024-02-24 13:05:59,368 INFO [train.py:471] (2/4) Epoch 879, batch 14, global_batch_idx: 32500, batch size: 95, loss[discriminator_loss=2.547, discriminator_real_loss=1.203, discriminator_fake_loss=1.344, generator_loss=30.77, generator_mel_loss=20.06, generator_kl_loss=2.084, generator_dur_loss=1.456, generator_adv_loss=2.258, generator_feat_match_loss=4.918, over 95.00 samples.], tot_loss[discriminator_loss=2.478, discriminator_real_loss=1.251, discriminator_fake_loss=1.227, generator_loss=30.68, generator_mel_loss=19.81, generator_kl_loss=2.012, generator_dur_loss=1.469, generator_adv_loss=2.35, generator_feat_match_loss=5.042, over 1055.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:08:02,257 INFO [train.py:845] (2/4) Start epoch 880
2024-02-24 13:10:44,715 INFO [train.py:471] (2/4) Epoch 880, batch 27, global_batch_idx: 32550, batch size: 59, loss[discriminator_loss=2.385, discriminator_real_loss=1.264, discriminator_fake_loss=1.121, generator_loss=31.05, generator_mel_loss=19.72, generator_kl_loss=1.938, generator_dur_loss=1.484, generator_adv_loss=2.5, generator_feat_match_loss=5.414, over 59.00 samples.], tot_loss[discriminator_loss=2.388, discriminator_real_loss=1.174, discriminator_fake_loss=1.214, generator_loss=30.65, generator_mel_loss=19.3, generator_kl_loss=1.96, generator_dur_loss=1.469, generator_adv_loss=2.476, generator_feat_match_loss=5.45, over 2060.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:11:31,057 INFO [train.py:845] (2/4) Start epoch 881
2024-02-24 13:14:49,971 INFO [train.py:845] (2/4) Start epoch 882
2024-02-24 13:15:16,945 INFO [train.py:471] (2/4) Epoch 882, batch 3, global_batch_idx: 32600, batch size: 52, loss[discriminator_loss=2.348, discriminator_real_loss=1.223, discriminator_fake_loss=1.124, generator_loss=30.96, generator_mel_loss=19.33, generator_kl_loss=1.992, generator_dur_loss=1.492, generator_adv_loss=2.438, generator_feat_match_loss=5.703, over 52.00 samples.], tot_loss[discriminator_loss=2.4, discriminator_real_loss=1.255, discriminator_fake_loss=1.144, generator_loss=30.6, generator_mel_loss=19.26, generator_kl_loss=1.956, generator_dur_loss=1.473, generator_adv_loss=2.54, generator_feat_match_loss=5.374, over 248.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:15:16,947 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 13:15:26,280 INFO [train.py:534] (2/4) Epoch 882, validation: discriminator_loss=2.327, discriminator_real_loss=0.9906, discriminator_fake_loss=1.336, generator_loss=31.93, generator_mel_loss=20.02, generator_kl_loss=2.166, generator_dur_loss=1.475, generator_adv_loss=2.37, generator_feat_match_loss=5.899, over 100.00 samples.
2024-02-24 13:15:26,281 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 13:18:30,406 INFO [train.py:845] (2/4) Start epoch 883
2024-02-24 13:20:03,116 INFO [train.py:471] (2/4) Epoch 883, batch 16, global_batch_idx: 32650, batch size: 49, loss[discriminator_loss=2.398, discriminator_real_loss=1.188, discriminator_fake_loss=1.211, generator_loss=30.54, generator_mel_loss=19.28, generator_kl_loss=1.99, generator_dur_loss=1.475, generator_adv_loss=2.512, generator_feat_match_loss=5.285, over 49.00 samples.], tot_loss[discriminator_loss=2.381, discriminator_real_loss=1.207, discriminator_fake_loss=1.174, generator_loss=30.56, generator_mel_loss=19.25, generator_kl_loss=1.971, generator_dur_loss=1.47, generator_adv_loss=2.466, generator_feat_match_loss=5.399, over 1133.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:21:50,911 INFO [train.py:845] (2/4) Start epoch 884
2024-02-24 13:24:36,070 INFO [train.py:471] (2/4) Epoch 884, batch 29, global_batch_idx: 32700, batch size: 79, loss[discriminator_loss=2.32, discriminator_real_loss=1.157, discriminator_fake_loss=1.163, generator_loss=31.44, generator_mel_loss=19.65, generator_kl_loss=1.934, generator_dur_loss=1.487, generator_adv_loss=2.543, generator_feat_match_loss=5.82, over 79.00 samples.], tot_loss[discriminator_loss=2.374, discriminator_real_loss=1.187, discriminator_fake_loss=1.188, generator_loss=30.8, generator_mel_loss=19.33, generator_kl_loss=1.97, generator_dur_loss=1.471, generator_adv_loss=2.489, generator_feat_match_loss=5.545, over 2093.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:25:21,476 INFO [train.py:845] (2/4) Start epoch 885
2024-02-24 13:28:52,821 INFO [train.py:845] (2/4) Start epoch 886
2024-02-24 13:29:34,534 INFO [train.py:471] (2/4) Epoch 886, batch 5, global_batch_idx: 32750, batch size: 85, loss[discriminator_loss=2.576, discriminator_real_loss=1.254, discriminator_fake_loss=1.322, generator_loss=29.9, generator_mel_loss=19.16, generator_kl_loss=1.97, generator_dur_loss=1.472, generator_adv_loss=2.281, generator_feat_match_loss=5.016, over 85.00 samples.], tot_loss[discriminator_loss=2.463, discriminator_real_loss=1.235, discriminator_fake_loss=1.228, generator_loss=30.57, generator_mel_loss=19.53, generator_kl_loss=1.982, generator_dur_loss=1.466, generator_adv_loss=2.408, generator_feat_match_loss=5.191, over 441.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:32:19,001 INFO [train.py:845] (2/4) Start epoch 887
2024-02-24 13:34:10,363 INFO [train.py:471] (2/4) Epoch 887, batch 18, global_batch_idx: 32800, batch size: 101, loss[discriminator_loss=2.27, discriminator_real_loss=1.124, discriminator_fake_loss=1.145, generator_loss=31.21, generator_mel_loss=19.08, generator_kl_loss=1.952, generator_dur_loss=1.459, generator_adv_loss=2.67, generator_feat_match_loss=6.051, over 101.00 samples.], tot_loss[discriminator_loss=2.334, discriminator_real_loss=1.178, discriminator_fake_loss=1.156, generator_loss=30.96, generator_mel_loss=19.3, generator_kl_loss=1.979, generator_dur_loss=1.468, generator_adv_loss=2.523, generator_feat_match_loss=5.688, over 1478.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 16.0
2024-02-24 13:34:10,365 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 13:34:19,058 INFO [train.py:534] (2/4) Epoch 887, validation: discriminator_loss=2.258, discriminator_real_loss=1.123, discriminator_fake_loss=1.135, generator_loss=32.18, generator_mel_loss=20.02, generator_kl_loss=2.092, generator_dur_loss=1.473, generator_adv_loss=2.536, generator_feat_match_loss=6.053, over 100.00 samples.
2024-02-24 13:34:19,059 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 13:35:55,276 INFO [train.py:845] (2/4) Start epoch 888
2024-02-24 13:38:41,293 INFO [train.py:471] (2/4) Epoch 888, batch 31, global_batch_idx: 32850, batch size: 81, loss[discriminator_loss=2.225, discriminator_real_loss=1.326, discriminator_fake_loss=0.8984, generator_loss=31.27, generator_mel_loss=19.41, generator_kl_loss=1.964, generator_dur_loss=1.448, generator_adv_loss=2.422, generator_feat_match_loss=6.023, over 81.00 samples.], tot_loss[discriminator_loss=2.319, discriminator_real_loss=1.16, discriminator_fake_loss=1.159, generator_loss=30.86, generator_mel_loss=19.22, generator_kl_loss=1.991, generator_dur_loss=1.468, generator_adv_loss=2.522, generator_feat_match_loss=5.661, over 2143.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:39:16,415 INFO [train.py:845] (2/4) Start epoch 889
2024-02-24 13:42:44,445 INFO [train.py:845] (2/4) Start epoch 890
2024-02-24 13:43:36,172 INFO [train.py:471] (2/4) Epoch 890, batch 7, global_batch_idx: 32900, batch size: 110, loss[discriminator_loss=2.275, discriminator_real_loss=1.08, discriminator_fake_loss=1.195, generator_loss=30.75, generator_mel_loss=19.35, generator_kl_loss=1.899, generator_dur_loss=1.432, generator_adv_loss=2.559, generator_feat_match_loss=5.504, over 110.00 samples.], tot_loss[discriminator_loss=2.313, discriminator_real_loss=1.147, discriminator_fake_loss=1.166, generator_loss=30.57, generator_mel_loss=19.13, generator_kl_loss=2.006, generator_dur_loss=1.46, generator_adv_loss=2.501, generator_feat_match_loss=5.473, over 691.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:46:13,339 INFO [train.py:845] (2/4) Start epoch 891
2024-02-24 13:48:16,994 INFO [train.py:471] (2/4) Epoch 891, batch 20, global_batch_idx: 32950, batch size: 51, loss[discriminator_loss=2.289, discriminator_real_loss=1.146, discriminator_fake_loss=1.144, generator_loss=31.27, generator_mel_loss=19.42, generator_kl_loss=1.976, generator_dur_loss=1.486, generator_adv_loss=2.541, generator_feat_match_loss=5.844, over 51.00 samples.], tot_loss[discriminator_loss=2.341, discriminator_real_loss=1.196, discriminator_fake_loss=1.145, generator_loss=30.83, generator_mel_loss=19.25, generator_kl_loss=1.964, generator_dur_loss=1.462, generator_adv_loss=2.538, generator_feat_match_loss=5.625, over 1514.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:49:48,927 INFO [train.py:845] (2/4) Start epoch 892
2024-02-24 13:53:02,765 INFO [train.py:471] (2/4) Epoch 892, batch 33, global_batch_idx: 33000, batch size: 69, loss[discriminator_loss=2.316, discriminator_real_loss=1.139, discriminator_fake_loss=1.179, generator_loss=30.21, generator_mel_loss=18.84, generator_kl_loss=1.983, generator_dur_loss=1.439, generator_adv_loss=2.551, generator_feat_match_loss=5.398, over 69.00 samples.], tot_loss[discriminator_loss=2.353, discriminator_real_loss=1.186, discriminator_fake_loss=1.168, generator_loss=30.83, generator_mel_loss=19.22, generator_kl_loss=1.964, generator_dur_loss=1.463, generator_adv_loss=2.527, generator_feat_match_loss=5.657, over 2464.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 13:53:02,767 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 13:53:11,546 INFO [train.py:534] (2/4) Epoch 892, validation: discriminator_loss=2.249, discriminator_real_loss=1.092, discriminator_fake_loss=1.157, generator_loss=32.05, generator_mel_loss=19.98, generator_kl_loss=2.083, generator_dur_loss=1.472, generator_adv_loss=2.481, generator_feat_match_loss=6.025, over 100.00 samples.
2024-02-24 13:53:11,547 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 13:53:27,060 INFO [train.py:845] (2/4) Start epoch 893
2024-02-24 13:56:56,835 INFO [train.py:845] (2/4) Start epoch 894
2024-02-24 13:57:53,609 INFO [train.py:471] (2/4) Epoch 894, batch 9, global_batch_idx: 33050, batch size: 61, loss[discriminator_loss=2.285, discriminator_real_loss=1.314, discriminator_fake_loss=0.9707, generator_loss=30.66, generator_mel_loss=18.91, generator_kl_loss=1.968, generator_dur_loss=1.472, generator_adv_loss=2.564, generator_feat_match_loss=5.746, over 61.00 samples.], tot_loss[discriminator_loss=2.328, discriminator_real_loss=1.224, discriminator_fake_loss=1.104, generator_loss=31.08, generator_mel_loss=19.09, generator_kl_loss=1.958, generator_dur_loss=1.461, generator_adv_loss=2.609, generator_feat_match_loss=5.962, over 693.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:00:27,307 INFO [train.py:845] (2/4) Start epoch 895
2024-02-24 14:02:38,229 INFO [train.py:471] (2/4) Epoch 895, batch 22, global_batch_idx: 33100, batch size: 52, loss[discriminator_loss=2.516, discriminator_real_loss=1.22, discriminator_fake_loss=1.296, generator_loss=30.7, generator_mel_loss=19.7, generator_kl_loss=1.903, generator_dur_loss=1.452, generator_adv_loss=2.547, generator_feat_match_loss=5.098, over 52.00 samples.], tot_loss[discriminator_loss=2.441, discriminator_real_loss=1.232, discriminator_fake_loss=1.209, generator_loss=30.58, generator_mel_loss=19.38, generator_kl_loss=1.959, generator_dur_loss=1.461, generator_adv_loss=2.465, generator_feat_match_loss=5.315, over 1615.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:03:57,573 INFO [train.py:845] (2/4) Start epoch 896
2024-02-24 14:07:24,293 INFO [train.py:471] (2/4) Epoch 896, batch 35, global_batch_idx: 33150, batch size: 85, loss[discriminator_loss=2.387, discriminator_real_loss=1.202, discriminator_fake_loss=1.184, generator_loss=29.7, generator_mel_loss=18.72, generator_kl_loss=1.917, generator_dur_loss=1.463, generator_adv_loss=2.613, generator_feat_match_loss=4.984, over 85.00 samples.], tot_loss[discriminator_loss=2.351, discriminator_real_loss=1.18, discriminator_fake_loss=1.171, generator_loss=30.67, generator_mel_loss=19.18, generator_kl_loss=1.972, generator_dur_loss=1.465, generator_adv_loss=2.503, generator_feat_match_loss=5.554, over 2731.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:07:28,542 INFO [train.py:845] (2/4) Start epoch 897
2024-02-24 14:10:57,467 INFO [train.py:845] (2/4) Start epoch 898
2024-02-24 14:12:11,331 INFO [train.py:471] (2/4) Epoch 898, batch 11, global_batch_idx: 33200, batch size: 67, loss[discriminator_loss=2.354, discriminator_real_loss=1.177, discriminator_fake_loss=1.177, generator_loss=30.95, generator_mel_loss=19.27, generator_kl_loss=2.01, generator_dur_loss=1.463, generator_adv_loss=2.496, generator_feat_match_loss=5.711, over 67.00 samples.], tot_loss[discriminator_loss=2.43, discriminator_real_loss=1.245, discriminator_fake_loss=1.185, generator_loss=30.97, generator_mel_loss=19.34, generator_kl_loss=1.981, generator_dur_loss=1.464, generator_adv_loss=2.548, generator_feat_match_loss=5.636, over 971.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 16.0
2024-02-24 14:12:11,332 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 14:12:20,141 INFO [train.py:534] (2/4) Epoch 898, validation: discriminator_loss=2.307, discriminator_real_loss=1.142, discriminator_fake_loss=1.165, generator_loss=31.75, generator_mel_loss=20.1, generator_kl_loss=2.145, generator_dur_loss=1.465, generator_adv_loss=2.441, generator_feat_match_loss=5.593, over 100.00 samples.
2024-02-24 14:12:20,142 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 14:14:33,080 INFO [train.py:845] (2/4) Start epoch 899
2024-02-24 14:17:00,806 INFO [train.py:471] (2/4) Epoch 899, batch 24, global_batch_idx: 33250, batch size: 79, loss[discriminator_loss=2.461, discriminator_real_loss=1.08, discriminator_fake_loss=1.38, generator_loss=30.23, generator_mel_loss=19.01, generator_kl_loss=1.92, generator_dur_loss=1.459, generator_adv_loss=2.26, generator_feat_match_loss=5.574, over 79.00 samples.], tot_loss[discriminator_loss=2.366, discriminator_real_loss=1.187, discriminator_fake_loss=1.178, generator_loss=30.94, generator_mel_loss=19.17, generator_kl_loss=1.957, generator_dur_loss=1.465, generator_adv_loss=2.566, generator_feat_match_loss=5.779, over 1716.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:18:05,959 INFO [train.py:845] (2/4) Start epoch 900
2024-02-24 14:21:33,274 INFO [train.py:845] (2/4) Start epoch 901
2024-02-24 14:21:45,409 INFO [train.py:471] (2/4) Epoch 901, batch 0, global_batch_idx: 33300, batch size: 81, loss[discriminator_loss=2.408, discriminator_real_loss=1.289, discriminator_fake_loss=1.119, generator_loss=29.9, generator_mel_loss=18.84, generator_kl_loss=1.892, generator_dur_loss=1.465, generator_adv_loss=2.49, generator_feat_match_loss=5.211, over 81.00 samples.], tot_loss[discriminator_loss=2.408, discriminator_real_loss=1.289, discriminator_fake_loss=1.119, generator_loss=29.9, generator_mel_loss=18.84, generator_kl_loss=1.892, generator_dur_loss=1.465, generator_adv_loss=2.49, generator_feat_match_loss=5.211, over 81.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:24:59,114 INFO [train.py:845] (2/4) Start epoch 902
2024-02-24 14:26:17,697 INFO [train.py:471] (2/4) Epoch 902, batch 13, global_batch_idx: 33350, batch size: 50, loss[discriminator_loss=2.699, discriminator_real_loss=1.354, discriminator_fake_loss=1.344, generator_loss=29.23, generator_mel_loss=19.47, generator_kl_loss=1.976, generator_dur_loss=1.486, generator_adv_loss=2.215, generator_feat_match_loss=4.082, over 50.00 samples.], tot_loss[discriminator_loss=2.505, discriminator_real_loss=1.259, discriminator_fake_loss=1.246, generator_loss=30.61, generator_mel_loss=19.8, generator_kl_loss=2.02, generator_dur_loss=1.476, generator_adv_loss=2.359, generator_feat_match_loss=4.951, over 887.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:28:30,424 INFO [train.py:845] (2/4) Start epoch 903
2024-02-24 14:30:59,292 INFO [train.py:471] (2/4) Epoch 903, batch 26, global_batch_idx: 33400, batch size: 85, loss[discriminator_loss=2.348, discriminator_real_loss=1.213, discriminator_fake_loss=1.135, generator_loss=29.88, generator_mel_loss=18.86, generator_kl_loss=1.987, generator_dur_loss=1.464, generator_adv_loss=2.465, generator_feat_match_loss=5.109, over 85.00 samples.], tot_loss[discriminator_loss=2.43, discriminator_real_loss=1.239, discriminator_fake_loss=1.19, generator_loss=30.36, generator_mel_loss=19.1, generator_kl_loss=1.948, generator_dur_loss=1.465, generator_adv_loss=2.496, generator_feat_match_loss=5.355, over 1961.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:30:59,294 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 14:31:08,285 INFO [train.py:534] (2/4) Epoch 903, validation: discriminator_loss=2.417, discriminator_real_loss=1.134, discriminator_fake_loss=1.283, generator_loss=30.38, generator_mel_loss=19.67, generator_kl_loss=2.172, generator_dur_loss=1.468, generator_adv_loss=2.054, generator_feat_match_loss=5.017, over 100.00 samples.
2024-02-24 14:31:08,286 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 14:32:04,205 INFO [train.py:845] (2/4) Start epoch 904
2024-02-24 14:35:33,419 INFO [train.py:845] (2/4) Start epoch 905
2024-02-24 14:35:53,346 INFO [train.py:471] (2/4) Epoch 905, batch 2, global_batch_idx: 33450, batch size: 55, loss[discriminator_loss=2.393, discriminator_real_loss=1.141, discriminator_fake_loss=1.252, generator_loss=30.84, generator_mel_loss=19.22, generator_kl_loss=1.984, generator_dur_loss=1.505, generator_adv_loss=2.516, generator_feat_match_loss=5.617, over 55.00 samples.], tot_loss[discriminator_loss=2.287, discriminator_real_loss=1.116, discriminator_fake_loss=1.171, generator_loss=30.91, generator_mel_loss=19.15, generator_kl_loss=1.973, generator_dur_loss=1.482, generator_adv_loss=2.531, generator_feat_match_loss=5.774, over 174.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:38:58,785 INFO [train.py:845] (2/4) Start epoch 906
2024-02-24 14:40:35,458 INFO [train.py:471] (2/4) Epoch 906, batch 15, global_batch_idx: 33500, batch size: 60, loss[discriminator_loss=2.309, discriminator_real_loss=1.12, discriminator_fake_loss=1.189, generator_loss=31.23, generator_mel_loss=19.61, generator_kl_loss=2.058, generator_dur_loss=1.47, generator_adv_loss=2.357, generator_feat_match_loss=5.73, over 60.00 samples.], tot_loss[discriminator_loss=2.336, discriminator_real_loss=1.172, discriminator_fake_loss=1.164, generator_loss=30.87, generator_mel_loss=19.3, generator_kl_loss=1.966, generator_dur_loss=1.466, generator_adv_loss=2.502, generator_feat_match_loss=5.63, over 1093.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:42:25,920 INFO [train.py:845] (2/4) Start epoch 907
2024-02-24 14:45:17,456 INFO [train.py:471] (2/4) Epoch 907, batch 28, global_batch_idx: 33550, batch size: 59, loss[discriminator_loss=2.305, discriminator_real_loss=1.16, discriminator_fake_loss=1.144, generator_loss=31.15, generator_mel_loss=19.55, generator_kl_loss=2.012, generator_dur_loss=1.455, generator_adv_loss=2.566, generator_feat_match_loss=5.566, over 59.00 samples.], tot_loss[discriminator_loss=2.422, discriminator_real_loss=1.218, discriminator_fake_loss=1.204, generator_loss=30.57, generator_mel_loss=19.34, generator_kl_loss=1.958, generator_dur_loss=1.467, generator_adv_loss=2.446, generator_feat_match_loss=5.357, over 2039.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:45:55,195 INFO [train.py:845] (2/4) Start epoch 908
2024-02-24 14:49:21,585 INFO [train.py:845] (2/4) Start epoch 909
2024-02-24 14:49:55,824 INFO [train.py:471] (2/4) Epoch 909, batch 4, global_batch_idx: 33600, batch size: 95, loss[discriminator_loss=2.324, discriminator_real_loss=1.169, discriminator_fake_loss=1.154, generator_loss=30.94, generator_mel_loss=19.29, generator_kl_loss=1.924, generator_dur_loss=1.45, generator_adv_loss=2.623, generator_feat_match_loss=5.648, over 95.00 samples.], tot_loss[discriminator_loss=2.308, discriminator_real_loss=1.217, discriminator_fake_loss=1.092, generator_loss=30.86, generator_mel_loss=19.12, generator_kl_loss=1.95, generator_dur_loss=1.452, generator_adv_loss=2.578, generator_feat_match_loss=5.759, over 392.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 16.0
2024-02-24 14:49:55,826 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 14:50:04,830 INFO [train.py:534] (2/4) Epoch 909, validation: discriminator_loss=2.548, discriminator_real_loss=1.19, discriminator_fake_loss=1.359, generator_loss=31.49, generator_mel_loss=20.29, generator_kl_loss=2.146, generator_dur_loss=1.466, generator_adv_loss=2.153, generator_feat_match_loss=5.436, over 100.00 samples.
2024-02-24 14:50:04,831 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 14:53:01,615 INFO [train.py:845] (2/4) Start epoch 910
2024-02-24 14:54:49,492 INFO [train.py:471] (2/4) Epoch 910, batch 17, global_batch_idx: 33650, batch size: 90, loss[discriminator_loss=2.393, discriminator_real_loss=1.172, discriminator_fake_loss=1.221, generator_loss=30.51, generator_mel_loss=19.3, generator_kl_loss=2.045, generator_dur_loss=1.451, generator_adv_loss=2.4, generator_feat_match_loss=5.316, over 90.00 samples.], tot_loss[discriminator_loss=2.38, discriminator_real_loss=1.203, discriminator_fake_loss=1.176, generator_loss=30.43, generator_mel_loss=19.16, generator_kl_loss=1.975, generator_dur_loss=1.463, generator_adv_loss=2.445, generator_feat_match_loss=5.386, over 1309.00 samples.], cur_lr_g: 1.79e-04, cur_lr_d: 1.79e-04, grad_scale: 8.0
2024-02-24 14:56:31,450 INFO [train.py:845] (2/4) Start epoch 911
2024-02-24 14:59:31,445 INFO [train.py:471] (2/4) Epoch 911, batch 30, global_batch_idx: 33700, batch size: 90, loss[discriminator_loss=2.471, discriminator_real_loss=1.327, discriminator_fake_loss=1.144, generator_loss=31.1, generator_mel_loss=19.47, generator_kl_loss=1.974, generator_dur_loss=1.455, generator_adv_loss=2.602, generator_feat_match_loss=5.602, over 90.00 samples.], tot_loss[discriminator_loss=2.475, discriminator_real_loss=1.232, discriminator_fake_loss=1.244, generator_loss=30.77, generator_mel_loss=19.53, generator_kl_loss=1.992, generator_dur_loss=1.465, generator_adv_loss=2.447, generator_feat_match_loss=5.335, over 2537.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:00:05,284 INFO [train.py:845] (2/4) Start epoch 912
2024-02-24 15:03:29,660 INFO [train.py:845] (2/4) Start epoch 913
2024-02-24 15:04:14,035 INFO [train.py:471] (2/4) Epoch 913, batch 6, global_batch_idx: 33750, batch size: 54, loss[discriminator_loss=2.188, discriminator_real_loss=1.069, discriminator_fake_loss=1.117, generator_loss=31.19, generator_mel_loss=18.94, generator_kl_loss=2.053, generator_dur_loss=1.5, generator_adv_loss=2.66, generator_feat_match_loss=6.035, over 54.00 samples.], tot_loss[discriminator_loss=2.35, discriminator_real_loss=1.169, discriminator_fake_loss=1.181, generator_loss=30.8, generator_mel_loss=19.14, generator_kl_loss=1.992, generator_dur_loss=1.464, generator_adv_loss=2.505, generator_feat_match_loss=5.701, over 428.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:06:55,838 INFO [train.py:845] (2/4) Start epoch 914
2024-02-24 15:08:58,307 INFO [train.py:471] (2/4) Epoch 914, batch 19, global_batch_idx: 33800, batch size: 69, loss[discriminator_loss=2.439, discriminator_real_loss=1.288, discriminator_fake_loss=1.151, generator_loss=30.72, generator_mel_loss=19.61, generator_kl_loss=2.065, generator_dur_loss=1.439, generator_adv_loss=2.475, generator_feat_match_loss=5.141, over 69.00 samples.], tot_loss[discriminator_loss=2.382, discriminator_real_loss=1.189, discriminator_fake_loss=1.193, generator_loss=30.46, generator_mel_loss=19.27, generator_kl_loss=1.985, generator_dur_loss=1.464, generator_adv_loss=2.414, generator_feat_match_loss=5.327, over 1437.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:08:58,309 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 15:09:07,053 INFO [train.py:534] (2/4) Epoch 914, validation: discriminator_loss=2.447, discriminator_real_loss=1.237, discriminator_fake_loss=1.21, generator_loss=31.6, generator_mel_loss=20.09, generator_kl_loss=2.2, generator_dur_loss=1.469, generator_adv_loss=2.405, generator_feat_match_loss=5.437, over 100.00 samples.
2024-02-24 15:09:07,054 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 15:10:33,176 INFO [train.py:845] (2/4) Start epoch 915
2024-02-24 15:13:32,294 INFO [train.py:471] (2/4) Epoch 915, batch 32, global_batch_idx: 33850, batch size: 101, loss[discriminator_loss=2.32, discriminator_real_loss=1.177, discriminator_fake_loss=1.145, generator_loss=31.45, generator_mel_loss=19.58, generator_kl_loss=2.093, generator_dur_loss=1.452, generator_adv_loss=2.637, generator_feat_match_loss=5.691, over 101.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.202, discriminator_fake_loss=1.199, generator_loss=30.86, generator_mel_loss=19.54, generator_kl_loss=2.006, generator_dur_loss=1.466, generator_adv_loss=2.44, generator_feat_match_loss=5.406, over 2483.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:14:00,328 INFO [train.py:845] (2/4) Start epoch 916
2024-02-24 15:17:33,355 INFO [train.py:845] (2/4) Start epoch 917
2024-02-24 15:18:25,049 INFO [train.py:471] (2/4) Epoch 917, batch 8, global_batch_idx: 33900, batch size: 76, loss[discriminator_loss=2.35, discriminator_real_loss=1.136, discriminator_fake_loss=1.214, generator_loss=30.51, generator_mel_loss=19.14, generator_kl_loss=1.972, generator_dur_loss=1.481, generator_adv_loss=2.369, generator_feat_match_loss=5.555, over 76.00 samples.], tot_loss[discriminator_loss=2.307, discriminator_real_loss=1.132, discriminator_fake_loss=1.175, generator_loss=30.72, generator_mel_loss=19.1, generator_kl_loss=1.987, generator_dur_loss=1.465, generator_adv_loss=2.495, generator_feat_match_loss=5.673, over 668.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:20:58,427 INFO [train.py:845] (2/4) Start epoch 918
2024-02-24 15:23:06,467 INFO [train.py:471] (2/4) Epoch 918, batch 21, global_batch_idx: 33950, batch size: 52, loss[discriminator_loss=2.188, discriminator_real_loss=1.063, discriminator_fake_loss=1.123, generator_loss=31.21, generator_mel_loss=19.12, generator_kl_loss=1.957, generator_dur_loss=1.493, generator_adv_loss=2.652, generator_feat_match_loss=5.992, over 52.00 samples.], tot_loss[discriminator_loss=2.338, discriminator_real_loss=1.171, discriminator_fake_loss=1.167, generator_loss=30.86, generator_mel_loss=19.09, generator_kl_loss=1.973, generator_dur_loss=1.465, generator_adv_loss=2.54, generator_feat_match_loss=5.784, over 1615.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:24:28,409 INFO [train.py:845] (2/4) Start epoch 919
2024-02-24 15:27:44,027 INFO [train.py:471] (2/4) Epoch 919, batch 34, global_batch_idx: 34000, batch size: 59, loss[discriminator_loss=2.23, discriminator_real_loss=1.03, discriminator_fake_loss=1.201, generator_loss=31.23, generator_mel_loss=19.36, generator_kl_loss=2.023, generator_dur_loss=1.486, generator_adv_loss=2.518, generator_feat_match_loss=5.844, over 59.00 samples.], tot_loss[discriminator_loss=2.345, discriminator_real_loss=1.19, discriminator_fake_loss=1.155, generator_loss=30.55, generator_mel_loss=18.98, generator_kl_loss=1.962, generator_dur_loss=1.463, generator_adv_loss=2.516, generator_feat_match_loss=5.634, over 2629.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 16.0
2024-02-24 15:27:44,028 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 15:27:53,618 INFO [train.py:534] (2/4) Epoch 919, validation: discriminator_loss=2.372, discriminator_real_loss=1.08, discriminator_fake_loss=1.292, generator_loss=31.4, generator_mel_loss=19.76, generator_kl_loss=2.195, generator_dur_loss=1.476, generator_adv_loss=2.246, generator_feat_match_loss=5.725, over 100.00 samples.
2024-02-24 15:27:53,619 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 15:28:04,665 INFO [train.py:845] (2/4) Start epoch 920
2024-02-24 15:31:31,444 INFO [train.py:845] (2/4) Start epoch 921
2024-02-24 15:32:38,469 INFO [train.py:471] (2/4) Epoch 921, batch 10, global_batch_idx: 34050, batch size: 71, loss[discriminator_loss=2.684, discriminator_real_loss=1.193, discriminator_fake_loss=1.489, generator_loss=30.2, generator_mel_loss=19.17, generator_kl_loss=1.966, generator_dur_loss=1.452, generator_adv_loss=2.375, generator_feat_match_loss=5.234, over 71.00 samples.], tot_loss[discriminator_loss=2.341, discriminator_real_loss=1.213, discriminator_fake_loss=1.128, generator_loss=30.77, generator_mel_loss=18.93, generator_kl_loss=1.966, generator_dur_loss=1.467, generator_adv_loss=2.591, generator_feat_match_loss=5.818, over 783.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 16.0
2024-02-24 15:34:59,422 INFO [train.py:845] (2/4) Start epoch 922
2024-02-24 15:37:14,146 INFO [train.py:471] (2/4) Epoch 922, batch 23, global_batch_idx: 34100, batch size: 154, loss[discriminator_loss=2.277, discriminator_real_loss=1.198, discriminator_fake_loss=1.078, generator_loss=31.31, generator_mel_loss=19.39, generator_kl_loss=1.947, generator_dur_loss=1.415, generator_adv_loss=2.57, generator_feat_match_loss=5.992, over 154.00 samples.], tot_loss[discriminator_loss=2.321, discriminator_real_loss=1.172, discriminator_fake_loss=1.148, generator_loss=30.79, generator_mel_loss=19.09, generator_kl_loss=1.978, generator_dur_loss=1.461, generator_adv_loss=2.529, generator_feat_match_loss=5.738, over 1840.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 16.0
2024-02-24 15:38:23,842 INFO [train.py:845] (2/4) Start epoch 923
2024-02-24 15:41:50,062 INFO [train.py:471] (2/4) Epoch 923, batch 36, global_batch_idx: 34150, batch size: 65, loss[discriminator_loss=2.316, discriminator_real_loss=1.136, discriminator_fake_loss=1.181, generator_loss=30.55, generator_mel_loss=19.07, generator_kl_loss=1.895, generator_dur_loss=1.452, generator_adv_loss=2.51, generator_feat_match_loss=5.625, over 65.00 samples.], tot_loss[discriminator_loss=2.355, discriminator_real_loss=1.193, discriminator_fake_loss=1.162, generator_loss=30.72, generator_mel_loss=19.03, generator_kl_loss=1.977, generator_dur_loss=1.47, generator_adv_loss=2.544, generator_feat_match_loss=5.69, over 2457.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 16.0
2024-02-24 15:41:50,477 INFO [train.py:845] (2/4) Start epoch 924
2024-02-24 15:45:18,069 INFO [train.py:845] (2/4) Start epoch 925
2024-02-24 15:46:35,571 INFO [train.py:471] (2/4) Epoch 925, batch 12, global_batch_idx: 34200, batch size: 81, loss[discriminator_loss=2.33, discriminator_real_loss=1.195, discriminator_fake_loss=1.135, generator_loss=30.98, generator_mel_loss=19.21, generator_kl_loss=1.981, generator_dur_loss=1.467, generator_adv_loss=2.562, generator_feat_match_loss=5.754, over 81.00 samples.], tot_loss[discriminator_loss=2.372, discriminator_real_loss=1.192, discriminator_fake_loss=1.18, generator_loss=30.76, generator_mel_loss=19.24, generator_kl_loss=1.98, generator_dur_loss=1.475, generator_adv_loss=2.501, generator_feat_match_loss=5.559, over 907.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:46:35,573 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 15:46:44,217 INFO [train.py:534] (2/4) Epoch 925, validation: discriminator_loss=2.279, discriminator_real_loss=1.121, discriminator_fake_loss=1.158, generator_loss=31.41, generator_mel_loss=19.57, generator_kl_loss=2.116, generator_dur_loss=1.468, generator_adv_loss=2.477, generator_feat_match_loss=5.784, over 100.00 samples.
2024-02-24 15:46:44,217 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 15:48:52,469 INFO [train.py:845] (2/4) Start epoch 926
2024-02-24 15:51:22,603 INFO [train.py:471] (2/4) Epoch 926, batch 25, global_batch_idx: 34250, batch size: 56, loss[discriminator_loss=2.492, discriminator_real_loss=1.306, discriminator_fake_loss=1.188, generator_loss=31.25, generator_mel_loss=19.86, generator_kl_loss=1.985, generator_dur_loss=1.464, generator_adv_loss=2.484, generator_feat_match_loss=5.453, over 56.00 samples.], tot_loss[discriminator_loss=2.5, discriminator_real_loss=1.286, discriminator_fake_loss=1.214, generator_loss=30.58, generator_mel_loss=19.62, generator_kl_loss=1.981, generator_dur_loss=1.465, generator_adv_loss=2.383, generator_feat_match_loss=5.131, over 1828.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:52:22,993 INFO [train.py:845] (2/4) Start epoch 927
2024-02-24 15:55:51,763 INFO [train.py:845] (2/4) Start epoch 928
2024-02-24 15:56:11,522 INFO [train.py:471] (2/4) Epoch 928, batch 1, global_batch_idx: 34300, batch size: 85, loss[discriminator_loss=2.242, discriminator_real_loss=1.072, discriminator_fake_loss=1.169, generator_loss=31.58, generator_mel_loss=19.56, generator_kl_loss=1.9, generator_dur_loss=1.477, generator_adv_loss=2.676, generator_feat_match_loss=5.969, over 85.00 samples.], tot_loss[discriminator_loss=2.27, discriminator_real_loss=1.137, discriminator_fake_loss=1.133, generator_loss=31.32, generator_mel_loss=19.42, generator_kl_loss=1.939, generator_dur_loss=1.47, generator_adv_loss=2.581, generator_feat_match_loss=5.909, over 180.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 15:59:19,104 INFO [train.py:845] (2/4) Start epoch 929
2024-02-24 16:00:46,610 INFO [train.py:471] (2/4) Epoch 929, batch 14, global_batch_idx: 34350, batch size: 58, loss[discriminator_loss=2.502, discriminator_real_loss=1.33, discriminator_fake_loss=1.172, generator_loss=30.77, generator_mel_loss=19.35, generator_kl_loss=1.992, generator_dur_loss=1.465, generator_adv_loss=2.611, generator_feat_match_loss=5.352, over 58.00 samples.], tot_loss[discriminator_loss=2.507, discriminator_real_loss=1.267, discriminator_fake_loss=1.239, generator_loss=30.44, generator_mel_loss=19.63, generator_kl_loss=1.981, generator_dur_loss=1.468, generator_adv_loss=2.355, generator_feat_match_loss=5.006, over 973.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 4.0
2024-02-24 16:02:48,830 INFO [train.py:845] (2/4) Start epoch 930
2024-02-24 16:05:25,077 INFO [train.py:471] (2/4) Epoch 930, batch 27, global_batch_idx: 34400, batch size: 55, loss[discriminator_loss=2.273, discriminator_real_loss=1.22, discriminator_fake_loss=1.055, generator_loss=30.59, generator_mel_loss=18.81, generator_kl_loss=2.001, generator_dur_loss=1.488, generator_adv_loss=2.586, generator_feat_match_loss=5.703, over 55.00 samples.], tot_loss[discriminator_loss=2.354, discriminator_real_loss=1.191, discriminator_fake_loss=1.163, generator_loss=30.88, generator_mel_loss=19.14, generator_kl_loss=1.963, generator_dur_loss=1.465, generator_adv_loss=2.553, generator_feat_match_loss=5.764, over 2016.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:05:25,078 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 16:05:33,345 INFO [train.py:534] (2/4) Epoch 930, validation: discriminator_loss=2.232, discriminator_real_loss=1.029, discriminator_fake_loss=1.203, generator_loss=31.67, generator_mel_loss=19.64, generator_kl_loss=2.172, generator_dur_loss=1.464, generator_adv_loss=2.472, generator_feat_match_loss=5.926, over 100.00 samples.
2024-02-24 16:05:33,346 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 16:06:25,582 INFO [train.py:845] (2/4) Start epoch 931
2024-02-24 16:09:49,720 INFO [train.py:845] (2/4) Start epoch 932
2024-02-24 16:10:14,897 INFO [train.py:471] (2/4) Epoch 932, batch 3, global_batch_idx: 34450, batch size: 61, loss[discriminator_loss=2.416, discriminator_real_loss=1.274, discriminator_fake_loss=1.142, generator_loss=29.92, generator_mel_loss=18.62, generator_kl_loss=2.018, generator_dur_loss=1.459, generator_adv_loss=2.434, generator_feat_match_loss=5.387, over 61.00 samples.], tot_loss[discriminator_loss=2.416, discriminator_real_loss=1.23, discriminator_fake_loss=1.186, generator_loss=30.19, generator_mel_loss=18.96, generator_kl_loss=1.968, generator_dur_loss=1.465, generator_adv_loss=2.467, generator_feat_match_loss=5.332, over 257.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:13:08,837 INFO [train.py:845] (2/4) Start epoch 933
2024-02-24 16:14:54,119 INFO [train.py:471] (2/4) Epoch 933, batch 16, global_batch_idx: 34500, batch size: 51, loss[discriminator_loss=2.27, discriminator_real_loss=1.098, discriminator_fake_loss=1.172, generator_loss=31.54, generator_mel_loss=19.53, generator_kl_loss=1.887, generator_dur_loss=1.474, generator_adv_loss=2.611, generator_feat_match_loss=6.031, over 51.00 samples.], tot_loss[discriminator_loss=2.342, discriminator_real_loss=1.174, discriminator_fake_loss=1.168, generator_loss=30.76, generator_mel_loss=19.17, generator_kl_loss=1.975, generator_dur_loss=1.46, generator_adv_loss=2.5, generator_feat_match_loss=5.66, over 1312.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:16:39,820 INFO [train.py:845] (2/4) Start epoch 934
2024-02-24 16:19:27,693 INFO [train.py:471] (2/4) Epoch 934, batch 29, global_batch_idx: 34550, batch size: 63, loss[discriminator_loss=2.418, discriminator_real_loss=1.257, discriminator_fake_loss=1.16, generator_loss=30.62, generator_mel_loss=19.2, generator_kl_loss=1.925, generator_dur_loss=1.474, generator_adv_loss=2.404, generator_feat_match_loss=5.609, over 63.00 samples.], tot_loss[discriminator_loss=2.367, discriminator_real_loss=1.205, discriminator_fake_loss=1.162, generator_loss=30.62, generator_mel_loss=19, generator_kl_loss=1.961, generator_dur_loss=1.466, generator_adv_loss=2.552, generator_feat_match_loss=5.642, over 2081.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:20:06,547 INFO [train.py:845] (2/4) Start epoch 935
2024-02-24 16:23:31,242 INFO [train.py:845] (2/4) Start epoch 936
2024-02-24 16:24:07,279 INFO [train.py:471] (2/4) Epoch 936, batch 5, global_batch_idx: 34600, batch size: 95, loss[discriminator_loss=2.41, discriminator_real_loss=1.202, discriminator_fake_loss=1.208, generator_loss=30.95, generator_mel_loss=19.4, generator_kl_loss=1.97, generator_dur_loss=1.452, generator_adv_loss=2.498, generator_feat_match_loss=5.625, over 95.00 samples.], tot_loss[discriminator_loss=2.403, discriminator_real_loss=1.213, discriminator_fake_loss=1.189, generator_loss=30.46, generator_mel_loss=19.23, generator_kl_loss=1.967, generator_dur_loss=1.471, generator_adv_loss=2.425, generator_feat_match_loss=5.373, over 403.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:24:07,281 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 16:24:15,899 INFO [train.py:534] (2/4) Epoch 936, validation: discriminator_loss=2.492, discriminator_real_loss=1.208, discriminator_fake_loss=1.284, generator_loss=30.81, generator_mel_loss=19.98, generator_kl_loss=2.176, generator_dur_loss=1.474, generator_adv_loss=2.089, generator_feat_match_loss=5.091, over 100.00 samples.
2024-02-24 16:24:15,900 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 16:27:06,535 INFO [train.py:845] (2/4) Start epoch 937
2024-02-24 16:28:47,150 INFO [train.py:471] (2/4) Epoch 937, batch 18, global_batch_idx: 34650, batch size: 126, loss[discriminator_loss=2.723, discriminator_real_loss=1.348, discriminator_fake_loss=1.376, generator_loss=31, generator_mel_loss=19.27, generator_kl_loss=2.056, generator_dur_loss=1.44, generator_adv_loss=2.781, generator_feat_match_loss=5.457, over 126.00 samples.], tot_loss[discriminator_loss=2.355, discriminator_real_loss=1.169, discriminator_fake_loss=1.186, generator_loss=30.95, generator_mel_loss=19.2, generator_kl_loss=1.965, generator_dur_loss=1.469, generator_adv_loss=2.569, generator_feat_match_loss=5.748, over 1251.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:30:36,666 INFO [train.py:845] (2/4) Start epoch 938
2024-02-24 16:33:39,031 INFO [train.py:471] (2/4) Epoch 938, batch 31, global_batch_idx: 34700, batch size: 67, loss[discriminator_loss=2.355, discriminator_real_loss=1.312, discriminator_fake_loss=1.043, generator_loss=30.81, generator_mel_loss=19.04, generator_kl_loss=2.044, generator_dur_loss=1.48, generator_adv_loss=2.592, generator_feat_match_loss=5.648, over 67.00 samples.], tot_loss[discriminator_loss=2.365, discriminator_real_loss=1.194, discriminator_fake_loss=1.17, generator_loss=30.68, generator_mel_loss=19.07, generator_kl_loss=1.956, generator_dur_loss=1.464, generator_adv_loss=2.526, generator_feat_match_loss=5.663, over 2439.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:34:05,012 INFO [train.py:845] (2/4) Start epoch 939
2024-02-24 16:37:33,998 INFO [train.py:845] (2/4) Start epoch 940
2024-02-24 16:38:27,752 INFO [train.py:471] (2/4) Epoch 940, batch 7, global_batch_idx: 34750, batch size: 126, loss[discriminator_loss=2.395, discriminator_real_loss=1.346, discriminator_fake_loss=1.049, generator_loss=31.59, generator_mel_loss=19.96, generator_kl_loss=2.041, generator_dur_loss=1.449, generator_adv_loss=2.543, generator_feat_match_loss=5.598, over 126.00 samples.], tot_loss[discriminator_loss=2.432, discriminator_real_loss=1.261, discriminator_fake_loss=1.171, generator_loss=31.19, generator_mel_loss=19.84, generator_kl_loss=2.055, generator_dur_loss=1.458, generator_adv_loss=2.445, generator_feat_match_loss=5.391, over 757.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:41:03,827 INFO [train.py:845] (2/4) Start epoch 941
2024-02-24 16:43:04,331 INFO [train.py:471] (2/4) Epoch 941, batch 20, global_batch_idx: 34800, batch size: 61, loss[discriminator_loss=2.303, discriminator_real_loss=1.076, discriminator_fake_loss=1.227, generator_loss=31.03, generator_mel_loss=19.06, generator_kl_loss=1.959, generator_dur_loss=1.467, generator_adv_loss=2.604, generator_feat_match_loss=5.938, over 61.00 samples.], tot_loss[discriminator_loss=2.473, discriminator_real_loss=1.272, discriminator_fake_loss=1.2, generator_loss=30.49, generator_mel_loss=19.28, generator_kl_loss=1.991, generator_dur_loss=1.459, generator_adv_loss=2.448, generator_feat_match_loss=5.313, over 1563.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 16.0
2024-02-24 16:43:04,333 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 16:43:12,549 INFO [train.py:534] (2/4) Epoch 941, validation: discriminator_loss=2.856, discriminator_real_loss=1.061, discriminator_fake_loss=1.795, generator_loss=29.88, generator_mel_loss=19.8, generator_kl_loss=2.161, generator_dur_loss=1.477, generator_adv_loss=1.675, generator_feat_match_loss=4.763, over 100.00 samples.
2024-02-24 16:43:12,549 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 16:44:41,616 INFO [train.py:845] (2/4) Start epoch 942
2024-02-24 16:47:57,661 INFO [train.py:471] (2/4) Epoch 942, batch 33, global_batch_idx: 34850, batch size: 55, loss[discriminator_loss=2.494, discriminator_real_loss=1.294, discriminator_fake_loss=1.2, generator_loss=30.77, generator_mel_loss=19.71, generator_kl_loss=1.994, generator_dur_loss=1.492, generator_adv_loss=2.41, generator_feat_match_loss=5.156, over 55.00 samples.], tot_loss[discriminator_loss=2.401, discriminator_real_loss=1.226, discriminator_fake_loss=1.174, generator_loss=30.52, generator_mel_loss=19.19, generator_kl_loss=1.977, generator_dur_loss=1.472, generator_adv_loss=2.477, generator_feat_match_loss=5.41, over 2419.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:48:11,131 INFO [train.py:845] (2/4) Start epoch 943
2024-02-24 16:51:34,491 INFO [train.py:845] (2/4) Start epoch 944
2024-02-24 16:52:31,099 INFO [train.py:471] (2/4) Epoch 944, batch 9, global_batch_idx: 34900, batch size: 69, loss[discriminator_loss=2.293, discriminator_real_loss=1.128, discriminator_fake_loss=1.164, generator_loss=31.01, generator_mel_loss=19.12, generator_kl_loss=1.916, generator_dur_loss=1.481, generator_adv_loss=2.6, generator_feat_match_loss=5.887, over 69.00 samples.], tot_loss[discriminator_loss=2.354, discriminator_real_loss=1.187, discriminator_fake_loss=1.166, generator_loss=30.82, generator_mel_loss=19.21, generator_kl_loss=1.964, generator_dur_loss=1.47, generator_adv_loss=2.5, generator_feat_match_loss=5.673, over 769.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:55:04,602 INFO [train.py:845] (2/4) Start epoch 945
2024-02-24 16:57:13,542 INFO [train.py:471] (2/4) Epoch 945, batch 22, global_batch_idx: 34950, batch size: 56, loss[discriminator_loss=2.473, discriminator_real_loss=1.434, discriminator_fake_loss=1.039, generator_loss=31.07, generator_mel_loss=19.46, generator_kl_loss=2.012, generator_dur_loss=1.471, generator_adv_loss=2.564, generator_feat_match_loss=5.562, over 56.00 samples.], tot_loss[discriminator_loss=2.329, discriminator_real_loss=1.181, discriminator_fake_loss=1.149, generator_loss=30.73, generator_mel_loss=18.99, generator_kl_loss=1.969, generator_dur_loss=1.472, generator_adv_loss=2.559, generator_feat_match_loss=5.746, over 1479.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 16:58:30,231 INFO [train.py:845] (2/4) Start epoch 946
2024-02-24 17:01:55,565 INFO [train.py:471] (2/4) Epoch 946, batch 35, global_batch_idx: 35000, batch size: 64, loss[discriminator_loss=2.543, discriminator_real_loss=1.254, discriminator_fake_loss=1.288, generator_loss=30.71, generator_mel_loss=19.76, generator_kl_loss=1.952, generator_dur_loss=1.461, generator_adv_loss=2.355, generator_feat_match_loss=5.184, over 64.00 samples.], tot_loss[discriminator_loss=2.387, discriminator_real_loss=1.2, discriminator_fake_loss=1.188, generator_loss=30.63, generator_mel_loss=19.23, generator_kl_loss=1.977, generator_dur_loss=1.466, generator_adv_loss=2.462, generator_feat_match_loss=5.496, over 2472.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 17:01:55,567 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 17:02:04,505 INFO [train.py:534] (2/4) Epoch 946, validation: discriminator_loss=2.422, discriminator_real_loss=1.256, discriminator_fake_loss=1.166, generator_loss=31.43, generator_mel_loss=20.24, generator_kl_loss=2.194, generator_dur_loss=1.467, generator_adv_loss=2.341, generator_feat_match_loss=5.192, over 100.00 samples.
2024-02-24 17:02:04,505 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 17:02:10,743 INFO [train.py:845] (2/4) Start epoch 947
2024-02-24 17:05:37,449 INFO [train.py:845] (2/4) Start epoch 948
2024-02-24 17:06:48,263 INFO [train.py:471] (2/4) Epoch 948, batch 11, global_batch_idx: 35050, batch size: 59, loss[discriminator_loss=2.18, discriminator_real_loss=1.252, discriminator_fake_loss=0.9282, generator_loss=31.03, generator_mel_loss=19.02, generator_kl_loss=1.895, generator_dur_loss=1.475, generator_adv_loss=2.434, generator_feat_match_loss=6.211, over 59.00 samples.], tot_loss[discriminator_loss=2.395, discriminator_real_loss=1.158, discriminator_fake_loss=1.237, generator_loss=30.62, generator_mel_loss=19.11, generator_kl_loss=1.957, generator_dur_loss=1.462, generator_adv_loss=2.446, generator_feat_match_loss=5.648, over 899.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 17:09:05,005 INFO [train.py:845] (2/4) Start epoch 949
2024-02-24 17:11:34,808 INFO [train.py:471] (2/4) Epoch 949, batch 24, global_batch_idx: 35100, batch size: 56, loss[discriminator_loss=2.186, discriminator_real_loss=1.119, discriminator_fake_loss=1.066, generator_loss=31.16, generator_mel_loss=18.91, generator_kl_loss=1.905, generator_dur_loss=1.473, generator_adv_loss=2.664, generator_feat_match_loss=6.199, over 56.00 samples.], tot_loss[discriminator_loss=2.329, discriminator_real_loss=1.155, discriminator_fake_loss=1.175, generator_loss=30.69, generator_mel_loss=19.07, generator_kl_loss=1.985, generator_dur_loss=1.461, generator_adv_loss=2.497, generator_feat_match_loss=5.673, over 1922.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 17:12:38,358 INFO [train.py:845] (2/4) Start epoch 950
2024-02-24 17:16:10,900 INFO [train.py:845] (2/4) Start epoch 951
2024-02-24 17:16:24,177 INFO [train.py:471] (2/4) Epoch 951, batch 0, global_batch_idx: 35150, batch size: 55, loss[discriminator_loss=2.324, discriminator_real_loss=1.142, discriminator_fake_loss=1.182, generator_loss=30.46, generator_mel_loss=19.11, generator_kl_loss=2.029, generator_dur_loss=1.484, generator_adv_loss=2.391, generator_feat_match_loss=5.445, over 55.00 samples.], tot_loss[discriminator_loss=2.324, discriminator_real_loss=1.142, discriminator_fake_loss=1.182, generator_loss=30.46, generator_mel_loss=19.11, generator_kl_loss=2.029, generator_dur_loss=1.484, generator_adv_loss=2.391, generator_feat_match_loss=5.445, over 55.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 17:19:33,007 INFO [train.py:845] (2/4) Start epoch 952
2024-02-24 17:21:02,974 INFO [train.py:471] (2/4) Epoch 952, batch 13, global_batch_idx: 35200, batch size: 153, loss[discriminator_loss=2.266, discriminator_real_loss=1.137, discriminator_fake_loss=1.129, generator_loss=30.5, generator_mel_loss=18.84, generator_kl_loss=1.846, generator_dur_loss=1.447, generator_adv_loss=2.559, generator_feat_match_loss=5.812, over 153.00 samples.], tot_loss[discriminator_loss=2.326, discriminator_real_loss=1.185, discriminator_fake_loss=1.141, generator_loss=30.73, generator_mel_loss=19.03, generator_kl_loss=1.975, generator_dur_loss=1.459, generator_adv_loss=2.524, generator_feat_match_loss=5.74, over 1226.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 16.0
2024-02-24 17:21:02,976 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 17:21:11,547 INFO [train.py:534] (2/4) Epoch 952, validation: discriminator_loss=2.25, discriminator_real_loss=1.106, discriminator_fake_loss=1.143, generator_loss=31.22, generator_mel_loss=19.46, generator_kl_loss=2.107, generator_dur_loss=1.469, generator_adv_loss=2.479, generator_feat_match_loss=5.705, over 100.00 samples.
2024-02-24 17:21:11,548 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 17:23:08,272 INFO [train.py:845] (2/4) Start epoch 953
2024-02-24 17:25:39,398 INFO [train.py:471] (2/4) Epoch 953, batch 26, global_batch_idx: 35250, batch size: 60, loss[discriminator_loss=2.18, discriminator_real_loss=1.005, discriminator_fake_loss=1.176, generator_loss=31.18, generator_mel_loss=19.05, generator_kl_loss=1.934, generator_dur_loss=1.461, generator_adv_loss=2.66, generator_feat_match_loss=6.078, over 60.00 samples.], tot_loss[discriminator_loss=2.325, discriminator_real_loss=1.161, discriminator_fake_loss=1.164, generator_loss=30.85, generator_mel_loss=19.02, generator_kl_loss=1.997, generator_dur_loss=1.461, generator_adv_loss=2.55, generator_feat_match_loss=5.822, over 2069.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 17:26:33,547 INFO [train.py:845] (2/4) Start epoch 954
2024-02-24 17:30:04,826 INFO [train.py:845] (2/4) Start epoch 955
2024-02-24 17:30:27,647 INFO [train.py:471] (2/4) Epoch 955, batch 2, global_batch_idx: 35300, batch size: 55, loss[discriminator_loss=2.367, discriminator_real_loss=1.184, discriminator_fake_loss=1.185, generator_loss=30.63, generator_mel_loss=18.92, generator_kl_loss=1.985, generator_dur_loss=1.507, generator_adv_loss=2.592, generator_feat_match_loss=5.621, over 55.00 samples.], tot_loss[discriminator_loss=2.409, discriminator_real_loss=1.203, discriminator_fake_loss=1.206, generator_loss=30.64, generator_mel_loss=19.21, generator_kl_loss=1.989, generator_dur_loss=1.468, generator_adv_loss=2.405, generator_feat_match_loss=5.565, over 236.00 samples.], cur_lr_g: 1.78e-04, cur_lr_d: 1.78e-04, grad_scale: 8.0
2024-02-24 17:33:27,351 INFO [train.py:845] (2/4) Start epoch 956
2024-02-24 17:35:05,522 INFO [train.py:471] (2/4) Epoch 956, batch 15, global_batch_idx: 35350, batch size: 76, loss[discriminator_loss=2.629, discriminator_real_loss=1.592, discriminator_fake_loss=1.036, generator_loss=30.71, generator_mel_loss=19.21, generator_kl_loss=1.973, generator_dur_loss=1.461, generator_adv_loss=2.568, generator_feat_match_loss=5.5, over 76.00 samples.], tot_loss[discriminator_loss=2.363, discriminator_real_loss=1.174, discriminator_fake_loss=1.189, generator_loss=30.63, generator_mel_loss=18.86, generator_kl_loss=1.964, generator_dur_loss=1.472, generator_adv_loss=2.569, generator_feat_match_loss=5.771, over 1073.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 17:37:00,484 INFO [train.py:845] (2/4) Start epoch 957
2024-02-24 17:39:46,834 INFO [train.py:471] (2/4) Epoch 957, batch 28, global_batch_idx: 35400, batch size: 73, loss[discriminator_loss=2.34, discriminator_real_loss=1.172, discriminator_fake_loss=1.168, generator_loss=31.55, generator_mel_loss=19.84, generator_kl_loss=2.009, generator_dur_loss=1.453, generator_adv_loss=2.447, generator_feat_match_loss=5.797, over 73.00 samples.], tot_loss[discriminator_loss=2.363, discriminator_real_loss=1.188, discriminator_fake_loss=1.175, generator_loss=30.69, generator_mel_loss=19.22, generator_kl_loss=2.015, generator_dur_loss=1.465, generator_adv_loss=2.46, generator_feat_match_loss=5.533, over 2120.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 17:39:46,834 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 17:39:55,509 INFO [train.py:534] (2/4) Epoch 957, validation: discriminator_loss=2.405, discriminator_real_loss=1.142, discriminator_fake_loss=1.263, generator_loss=31.67, generator_mel_loss=20.33, generator_kl_loss=2.225, generator_dur_loss=1.474, generator_adv_loss=2.225, generator_feat_match_loss=5.418, over 100.00 samples.
2024-02-24 17:39:55,510 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 17:40:40,531 INFO [train.py:845] (2/4) Start epoch 958
2024-02-24 17:44:04,246 INFO [train.py:845] (2/4) Start epoch 959
2024-02-24 17:44:38,755 INFO [train.py:471] (2/4) Epoch 959, batch 4, global_batch_idx: 35450, batch size: 90, loss[discriminator_loss=2.326, discriminator_real_loss=1.396, discriminator_fake_loss=0.9312, generator_loss=31.66, generator_mel_loss=18.92, generator_kl_loss=1.96, generator_dur_loss=1.453, generator_adv_loss=2.898, generator_feat_match_loss=6.434, over 90.00 samples.], tot_loss[discriminator_loss=2.301, discriminator_real_loss=1.23, discriminator_fake_loss=1.071, generator_loss=31.41, generator_mel_loss=19.06, generator_kl_loss=1.974, generator_dur_loss=1.449, generator_adv_loss=2.661, generator_feat_match_loss=6.26, over 427.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 17:47:31,741 INFO [train.py:845] (2/4) Start epoch 960
2024-02-24 17:49:20,666 INFO [train.py:471] (2/4) Epoch 960, batch 17, global_batch_idx: 35500, batch size: 49, loss[discriminator_loss=2.438, discriminator_real_loss=1.229, discriminator_fake_loss=1.21, generator_loss=30.26, generator_mel_loss=19.14, generator_kl_loss=2.049, generator_dur_loss=1.469, generator_adv_loss=2.34, generator_feat_match_loss=5.262, over 49.00 samples.], tot_loss[discriminator_loss=2.422, discriminator_real_loss=1.236, discriminator_fake_loss=1.186, generator_loss=30.22, generator_mel_loss=19.16, generator_kl_loss=1.973, generator_dur_loss=1.462, generator_adv_loss=2.412, generator_feat_match_loss=5.212, over 1424.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 17:51:04,160 INFO [train.py:845] (2/4) Start epoch 961
2024-02-24 17:53:51,485 INFO [train.py:471] (2/4) Epoch 961, batch 30, global_batch_idx: 35550, batch size: 95, loss[discriminator_loss=2.383, discriminator_real_loss=1.257, discriminator_fake_loss=1.125, generator_loss=30.86, generator_mel_loss=19.34, generator_kl_loss=1.971, generator_dur_loss=1.44, generator_adv_loss=2.475, generator_feat_match_loss=5.629, over 95.00 samples.], tot_loss[discriminator_loss=2.357, discriminator_real_loss=1.187, discriminator_fake_loss=1.17, generator_loss=30.52, generator_mel_loss=19.01, generator_kl_loss=1.971, generator_dur_loss=1.467, generator_adv_loss=2.489, generator_feat_match_loss=5.58, over 2200.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 17:54:29,310 INFO [train.py:845] (2/4) Start epoch 962
2024-02-24 17:58:03,838 INFO [train.py:845] (2/4) Start epoch 963
2024-02-24 17:58:53,955 INFO [train.py:471] (2/4) Epoch 963, batch 6, global_batch_idx: 35600, batch size: 60, loss[discriminator_loss=2.355, discriminator_real_loss=1.206, discriminator_fake_loss=1.148, generator_loss=31.16, generator_mel_loss=19.73, generator_kl_loss=1.992, generator_dur_loss=1.493, generator_adv_loss=2.441, generator_feat_match_loss=5.504, over 60.00 samples.], tot_loss[discriminator_loss=2.43, discriminator_real_loss=1.233, discriminator_fake_loss=1.197, generator_loss=30.7, generator_mel_loss=19.59, generator_kl_loss=2.023, generator_dur_loss=1.469, generator_adv_loss=2.398, generator_feat_match_loss=5.214, over 471.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 16.0
2024-02-24 17:58:53,957 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 17:59:02,580 INFO [train.py:534] (2/4) Epoch 963, validation: discriminator_loss=2.424, discriminator_real_loss=1.14, discriminator_fake_loss=1.285, generator_loss=31.29, generator_mel_loss=20.13, generator_kl_loss=2.256, generator_dur_loss=1.465, generator_adv_loss=2.192, generator_feat_match_loss=5.247, over 100.00 samples.
2024-02-24 17:59:02,581 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 18:01:42,482 INFO [train.py:845] (2/4) Start epoch 964
2024-02-24 18:03:37,127 INFO [train.py:471] (2/4) Epoch 964, batch 19, global_batch_idx: 35650, batch size: 58, loss[discriminator_loss=2.58, discriminator_real_loss=1.154, discriminator_fake_loss=1.426, generator_loss=29.07, generator_mel_loss=18.68, generator_kl_loss=1.93, generator_dur_loss=1.455, generator_adv_loss=2.262, generator_feat_match_loss=4.738, over 58.00 samples.], tot_loss[discriminator_loss=2.561, discriminator_real_loss=1.317, discriminator_fake_loss=1.244, generator_loss=30.64, generator_mel_loss=19.14, generator_kl_loss=1.957, generator_dur_loss=1.465, generator_adv_loss=2.555, generator_feat_match_loss=5.522, over 1384.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 4.0
2024-02-24 18:05:11,146 INFO [train.py:845] (2/4) Start epoch 965
2024-02-24 18:08:23,596 INFO [train.py:471] (2/4) Epoch 965, batch 32, global_batch_idx: 35700, batch size: 69, loss[discriminator_loss=2.947, discriminator_real_loss=1.306, discriminator_fake_loss=1.642, generator_loss=28.95, generator_mel_loss=19.05, generator_kl_loss=2.002, generator_dur_loss=1.457, generator_adv_loss=1.939, generator_feat_match_loss=4.5, over 69.00 samples.], tot_loss[discriminator_loss=2.501, discriminator_real_loss=1.262, discriminator_fake_loss=1.239, generator_loss=30.45, generator_mel_loss=19.36, generator_kl_loss=1.981, generator_dur_loss=1.465, generator_adv_loss=2.413, generator_feat_match_loss=5.228, over 2374.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:08:43,119 INFO [train.py:845] (2/4) Start epoch 966
2024-02-24 18:12:10,274 INFO [train.py:845] (2/4) Start epoch 967
2024-02-24 18:13:09,643 INFO [train.py:471] (2/4) Epoch 967, batch 8, global_batch_idx: 35750, batch size: 53, loss[discriminator_loss=2.336, discriminator_real_loss=1.253, discriminator_fake_loss=1.082, generator_loss=30.91, generator_mel_loss=19.26, generator_kl_loss=1.974, generator_dur_loss=1.494, generator_adv_loss=2.562, generator_feat_match_loss=5.621, over 53.00 samples.], tot_loss[discriminator_loss=2.354, discriminator_real_loss=1.207, discriminator_fake_loss=1.146, generator_loss=30.58, generator_mel_loss=19.15, generator_kl_loss=1.985, generator_dur_loss=1.469, generator_adv_loss=2.502, generator_feat_match_loss=5.472, over 657.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:15:44,248 INFO [train.py:845] (2/4) Start epoch 968
2024-02-24 18:17:47,947 INFO [train.py:471] (2/4) Epoch 968, batch 21, global_batch_idx: 35800, batch size: 95, loss[discriminator_loss=2.254, discriminator_real_loss=1.122, discriminator_fake_loss=1.133, generator_loss=31.39, generator_mel_loss=19.51, generator_kl_loss=1.922, generator_dur_loss=1.447, generator_adv_loss=2.523, generator_feat_match_loss=5.996, over 95.00 samples.], tot_loss[discriminator_loss=2.518, discriminator_real_loss=1.291, discriminator_fake_loss=1.228, generator_loss=30.82, generator_mel_loss=19.71, generator_kl_loss=2.031, generator_dur_loss=1.46, generator_adv_loss=2.412, generator_feat_match_loss=5.207, over 1666.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:17:47,949 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 18:17:56,223 INFO [train.py:534] (2/4) Epoch 968, validation: discriminator_loss=2.295, discriminator_real_loss=1.019, discriminator_fake_loss=1.275, generator_loss=32.14, generator_mel_loss=20.07, generator_kl_loss=2.278, generator_dur_loss=1.466, generator_adv_loss=2.424, generator_feat_match_loss=5.902, over 100.00 samples.
2024-02-24 18:17:56,224 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 18:19:13,547 INFO [train.py:845] (2/4) Start epoch 969
2024-02-24 18:22:31,545 INFO [train.py:471] (2/4) Epoch 969, batch 34, global_batch_idx: 35850, batch size: 69, loss[discriminator_loss=2.273, discriminator_real_loss=1.107, discriminator_fake_loss=1.167, generator_loss=30.58, generator_mel_loss=18.98, generator_kl_loss=1.972, generator_dur_loss=1.45, generator_adv_loss=2.619, generator_feat_match_loss=5.566, over 69.00 samples.], tot_loss[discriminator_loss=2.417, discriminator_real_loss=1.215, discriminator_fake_loss=1.201, generator_loss=30.56, generator_mel_loss=19.21, generator_kl_loss=1.949, generator_dur_loss=1.459, generator_adv_loss=2.452, generator_feat_match_loss=5.488, over 2628.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:22:43,694 INFO [train.py:845] (2/4) Start epoch 970
2024-02-24 18:26:09,413 INFO [train.py:845] (2/4) Start epoch 971
2024-02-24 18:27:18,412 INFO [train.py:471] (2/4) Epoch 971, batch 10, global_batch_idx: 35900, batch size: 58, loss[discriminator_loss=2.488, discriminator_real_loss=1.19, discriminator_fake_loss=1.299, generator_loss=30.41, generator_mel_loss=19.38, generator_kl_loss=1.923, generator_dur_loss=1.462, generator_adv_loss=2.438, generator_feat_match_loss=5.211, over 58.00 samples.], tot_loss[discriminator_loss=2.493, discriminator_real_loss=1.253, discriminator_fake_loss=1.241, generator_loss=30.63, generator_mel_loss=19.7, generator_kl_loss=1.997, generator_dur_loss=1.468, generator_adv_loss=2.345, generator_feat_match_loss=5.127, over 795.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:29:37,265 INFO [train.py:845] (2/4) Start epoch 972
2024-02-24 18:31:56,871 INFO [train.py:471] (2/4) Epoch 972, batch 23, global_batch_idx: 35950, batch size: 51, loss[discriminator_loss=2.285, discriminator_real_loss=1.175, discriminator_fake_loss=1.111, generator_loss=30.79, generator_mel_loss=19.21, generator_kl_loss=2.026, generator_dur_loss=1.468, generator_adv_loss=2.359, generator_feat_match_loss=5.734, over 51.00 samples.], tot_loss[discriminator_loss=2.4, discriminator_real_loss=1.221, discriminator_fake_loss=1.179, generator_loss=30.81, generator_mel_loss=19.18, generator_kl_loss=1.967, generator_dur_loss=1.461, generator_adv_loss=2.537, generator_feat_match_loss=5.667, over 1643.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:33:07,941 INFO [train.py:845] (2/4) Start epoch 973
2024-02-24 18:36:35,426 INFO [train.py:471] (2/4) Epoch 973, batch 36, global_batch_idx: 36000, batch size: 110, loss[discriminator_loss=2.285, discriminator_real_loss=1.123, discriminator_fake_loss=1.162, generator_loss=31.18, generator_mel_loss=19.34, generator_kl_loss=2.078, generator_dur_loss=1.456, generator_adv_loss=2.504, generator_feat_match_loss=5.797, over 110.00 samples.], tot_loss[discriminator_loss=2.417, discriminator_real_loss=1.209, discriminator_fake_loss=1.208, generator_loss=30.44, generator_mel_loss=19.04, generator_kl_loss=1.979, generator_dur_loss=1.459, generator_adv_loss=2.47, generator_feat_match_loss=5.489, over 2773.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 16.0
2024-02-24 18:36:35,428 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 18:36:43,777 INFO [train.py:534] (2/4) Epoch 973, validation: discriminator_loss=2.281, discriminator_real_loss=1.118, discriminator_fake_loss=1.163, generator_loss=31.79, generator_mel_loss=19.75, generator_kl_loss=2.207, generator_dur_loss=1.475, generator_adv_loss=2.432, generator_feat_match_loss=5.929, over 100.00 samples.
2024-02-24 18:36:43,778 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 18:36:44,210 INFO [train.py:845] (2/4) Start epoch 974
2024-02-24 18:40:15,319 INFO [train.py:845] (2/4) Start epoch 975
2024-02-24 18:41:30,052 INFO [train.py:471] (2/4) Epoch 975, batch 12, global_batch_idx: 36050, batch size: 53, loss[discriminator_loss=2.512, discriminator_real_loss=1.298, discriminator_fake_loss=1.213, generator_loss=30.1, generator_mel_loss=19.42, generator_kl_loss=1.9, generator_dur_loss=1.48, generator_adv_loss=2.318, generator_feat_match_loss=4.977, over 53.00 samples.], tot_loss[discriminator_loss=2.447, discriminator_real_loss=1.222, discriminator_fake_loss=1.225, generator_loss=30.48, generator_mel_loss=19.52, generator_kl_loss=1.979, generator_dur_loss=1.468, generator_adv_loss=2.323, generator_feat_match_loss=5.186, over 941.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:43:44,648 INFO [train.py:845] (2/4) Start epoch 976
2024-02-24 18:46:14,290 INFO [train.py:471] (2/4) Epoch 976, batch 25, global_batch_idx: 36100, batch size: 58, loss[discriminator_loss=2.574, discriminator_real_loss=1.391, discriminator_fake_loss=1.185, generator_loss=30.48, generator_mel_loss=19.55, generator_kl_loss=1.942, generator_dur_loss=1.483, generator_adv_loss=2.525, generator_feat_match_loss=4.984, over 58.00 samples.], tot_loss[discriminator_loss=2.528, discriminator_real_loss=1.267, discriminator_fake_loss=1.262, generator_loss=30.38, generator_mel_loss=19.31, generator_kl_loss=1.971, generator_dur_loss=1.464, generator_adv_loss=2.396, generator_feat_match_loss=5.241, over 1835.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:47:14,830 INFO [train.py:845] (2/4) Start epoch 977
2024-02-24 18:50:43,734 INFO [train.py:845] (2/4) Start epoch 978
2024-02-24 18:51:02,920 INFO [train.py:471] (2/4) Epoch 978, batch 1, global_batch_idx: 36150, batch size: 73, loss[discriminator_loss=2.445, discriminator_real_loss=1.233, discriminator_fake_loss=1.211, generator_loss=30.69, generator_mel_loss=19.46, generator_kl_loss=2.007, generator_dur_loss=1.491, generator_adv_loss=2.389, generator_feat_match_loss=5.34, over 73.00 samples.], tot_loss[discriminator_loss=2.433, discriminator_real_loss=1.209, discriminator_fake_loss=1.223, generator_loss=30.58, generator_mel_loss=19.42, generator_kl_loss=2.016, generator_dur_loss=1.471, generator_adv_loss=2.34, generator_feat_match_loss=5.332, over 199.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:54:09,689 INFO [train.py:845] (2/4) Start epoch 979
2024-02-24 18:55:34,904 INFO [train.py:471] (2/4) Epoch 979, batch 14, global_batch_idx: 36200, batch size: 49, loss[discriminator_loss=2.332, discriminator_real_loss=1.193, discriminator_fake_loss=1.139, generator_loss=30.65, generator_mel_loss=18.92, generator_kl_loss=1.952, generator_dur_loss=1.463, generator_adv_loss=2.531, generator_feat_match_loss=5.785, over 49.00 samples.], tot_loss[discriminator_loss=2.392, discriminator_real_loss=1.226, discriminator_fake_loss=1.166, generator_loss=30.9, generator_mel_loss=19.14, generator_kl_loss=1.957, generator_dur_loss=1.461, generator_adv_loss=2.593, generator_feat_match_loss=5.751, over 1050.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 18:55:34,905 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 18:55:43,390 INFO [train.py:534] (2/4) Epoch 979, validation: discriminator_loss=2.281, discriminator_real_loss=1.066, discriminator_fake_loss=1.215, generator_loss=31.61, generator_mel_loss=19.61, generator_kl_loss=2.126, generator_dur_loss=1.469, generator_adv_loss=2.501, generator_feat_match_loss=5.898, over 100.00 samples.
2024-02-24 18:55:43,391 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 18:57:43,305 INFO [train.py:845] (2/4) Start epoch 980
2024-02-24 19:00:26,676 INFO [train.py:471] (2/4) Epoch 980, batch 27, global_batch_idx: 36250, batch size: 54, loss[discriminator_loss=2.438, discriminator_real_loss=1.188, discriminator_fake_loss=1.251, generator_loss=30.15, generator_mel_loss=18.58, generator_kl_loss=2.026, generator_dur_loss=1.497, generator_adv_loss=2.66, generator_feat_match_loss=5.391, over 54.00 samples.], tot_loss[discriminator_loss=2.38, discriminator_real_loss=1.208, discriminator_fake_loss=1.173, generator_loss=30.64, generator_mel_loss=19, generator_kl_loss=1.96, generator_dur_loss=1.463, generator_adv_loss=2.553, generator_feat_match_loss=5.665, over 2138.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:01:13,300 INFO [train.py:845] (2/4) Start epoch 981
2024-02-24 19:04:44,198 INFO [train.py:845] (2/4) Start epoch 982
2024-02-24 19:05:13,002 INFO [train.py:471] (2/4) Epoch 982, batch 3, global_batch_idx: 36300, batch size: 61, loss[discriminator_loss=2.342, discriminator_real_loss=1.258, discriminator_fake_loss=1.084, generator_loss=30.95, generator_mel_loss=19.44, generator_kl_loss=1.997, generator_dur_loss=1.478, generator_adv_loss=2.477, generator_feat_match_loss=5.555, over 61.00 samples.], tot_loss[discriminator_loss=2.35, discriminator_real_loss=1.258, discriminator_fake_loss=1.091, generator_loss=31, generator_mel_loss=19.45, generator_kl_loss=1.999, generator_dur_loss=1.468, generator_adv_loss=2.497, generator_feat_match_loss=5.587, over 311.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:08:13,951 INFO [train.py:845] (2/4) Start epoch 983
2024-02-24 19:09:55,376 INFO [train.py:471] (2/4) Epoch 983, batch 16, global_batch_idx: 36350, batch size: 154, loss[discriminator_loss=2.451, discriminator_real_loss=1.33, discriminator_fake_loss=1.121, generator_loss=30.79, generator_mel_loss=19.37, generator_kl_loss=1.84, generator_dur_loss=1.435, generator_adv_loss=2.607, generator_feat_match_loss=5.539, over 154.00 samples.], tot_loss[discriminator_loss=2.402, discriminator_real_loss=1.224, discriminator_fake_loss=1.177, generator_loss=30.81, generator_mel_loss=19.11, generator_kl_loss=1.975, generator_dur_loss=1.466, generator_adv_loss=2.558, generator_feat_match_loss=5.702, over 1196.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:11:44,595 INFO [train.py:845] (2/4) Start epoch 984
2024-02-24 19:14:34,343 INFO [train.py:471] (2/4) Epoch 984, batch 29, global_batch_idx: 36400, batch size: 55, loss[discriminator_loss=2.416, discriminator_real_loss=1.285, discriminator_fake_loss=1.131, generator_loss=29.56, generator_mel_loss=18.34, generator_kl_loss=2.011, generator_dur_loss=1.487, generator_adv_loss=2.496, generator_feat_match_loss=5.219, over 55.00 samples.], tot_loss[discriminator_loss=2.347, discriminator_real_loss=1.191, discriminator_fake_loss=1.157, generator_loss=30.65, generator_mel_loss=19, generator_kl_loss=1.979, generator_dur_loss=1.462, generator_adv_loss=2.528, generator_feat_match_loss=5.682, over 2134.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 16.0
2024-02-24 19:14:34,345 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 19:14:43,199 INFO [train.py:534] (2/4) Epoch 984, validation: discriminator_loss=2.285, discriminator_real_loss=1.101, discriminator_fake_loss=1.184, generator_loss=31.51, generator_mel_loss=19.53, generator_kl_loss=2.212, generator_dur_loss=1.46, generator_adv_loss=2.452, generator_feat_match_loss=5.857, over 100.00 samples.
2024-02-24 19:14:43,200 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 19:15:22,938 INFO [train.py:845] (2/4) Start epoch 985
2024-02-24 19:18:54,886 INFO [train.py:845] (2/4) Start epoch 986
2024-02-24 19:19:37,276 INFO [train.py:471] (2/4) Epoch 986, batch 5, global_batch_idx: 36450, batch size: 52, loss[discriminator_loss=2.656, discriminator_real_loss=1.247, discriminator_fake_loss=1.409, generator_loss=29.91, generator_mel_loss=19.11, generator_kl_loss=1.956, generator_dur_loss=1.477, generator_adv_loss=2.391, generator_feat_match_loss=4.969, over 52.00 samples.], tot_loss[discriminator_loss=2.403, discriminator_real_loss=1.189, discriminator_fake_loss=1.215, generator_loss=30.66, generator_mel_loss=19.01, generator_kl_loss=1.97, generator_dur_loss=1.46, generator_adv_loss=2.522, generator_feat_match_loss=5.697, over 413.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:22:19,331 INFO [train.py:845] (2/4) Start epoch 987
2024-02-24 19:24:16,744 INFO [train.py:471] (2/4) Epoch 987, batch 18, global_batch_idx: 36500, batch size: 64, loss[discriminator_loss=2.41, discriminator_real_loss=1.247, discriminator_fake_loss=1.164, generator_loss=30.21, generator_mel_loss=18.71, generator_kl_loss=1.947, generator_dur_loss=1.476, generator_adv_loss=2.559, generator_feat_match_loss=5.516, over 64.00 samples.], tot_loss[discriminator_loss=2.386, discriminator_real_loss=1.188, discriminator_fake_loss=1.198, generator_loss=30.37, generator_mel_loss=18.94, generator_kl_loss=1.958, generator_dur_loss=1.467, generator_adv_loss=2.461, generator_feat_match_loss=5.55, over 1339.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:25:47,605 INFO [train.py:845] (2/4) Start epoch 988
2024-02-24 19:28:54,683 INFO [train.py:471] (2/4) Epoch 988, batch 31, global_batch_idx: 36550, batch size: 53, loss[discriminator_loss=2.52, discriminator_real_loss=1.287, discriminator_fake_loss=1.231, generator_loss=31.07, generator_mel_loss=19.98, generator_kl_loss=2.002, generator_dur_loss=1.484, generator_adv_loss=2.449, generator_feat_match_loss=5.156, over 53.00 samples.], tot_loss[discriminator_loss=2.473, discriminator_real_loss=1.245, discriminator_fake_loss=1.228, generator_loss=30.49, generator_mel_loss=19.54, generator_kl_loss=1.984, generator_dur_loss=1.464, generator_adv_loss=2.34, generator_feat_match_loss=5.16, over 2243.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:29:18,958 INFO [train.py:845] (2/4) Start epoch 989
2024-02-24 19:32:50,218 INFO [train.py:845] (2/4) Start epoch 990
2024-02-24 19:33:40,821 INFO [train.py:471] (2/4) Epoch 990, batch 7, global_batch_idx: 36600, batch size: 59, loss[discriminator_loss=2.215, discriminator_real_loss=1.016, discriminator_fake_loss=1.199, generator_loss=31.18, generator_mel_loss=19.04, generator_kl_loss=2.069, generator_dur_loss=1.493, generator_adv_loss=2.578, generator_feat_match_loss=6.004, over 59.00 samples.], tot_loss[discriminator_loss=2.409, discriminator_real_loss=1.176, discriminator_fake_loss=1.232, generator_loss=30.73, generator_mel_loss=19.05, generator_kl_loss=1.983, generator_dur_loss=1.474, generator_adv_loss=2.484, generator_feat_match_loss=5.737, over 498.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:33:40,823 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 19:33:49,516 INFO [train.py:534] (2/4) Epoch 990, validation: discriminator_loss=2.207, discriminator_real_loss=1.045, discriminator_fake_loss=1.162, generator_loss=31.92, generator_mel_loss=19.62, generator_kl_loss=2.194, generator_dur_loss=1.468, generator_adv_loss=2.473, generator_feat_match_loss=6.168, over 100.00 samples.
2024-02-24 19:33:49,517 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 19:36:30,932 INFO [train.py:845] (2/4) Start epoch 991
2024-02-24 19:38:36,931 INFO [train.py:471] (2/4) Epoch 991, batch 20, global_batch_idx: 36650, batch size: 76, loss[discriminator_loss=2.33, discriminator_real_loss=1.233, discriminator_fake_loss=1.097, generator_loss=30.27, generator_mel_loss=18.83, generator_kl_loss=1.888, generator_dur_loss=1.463, generator_adv_loss=2.531, generator_feat_match_loss=5.562, over 76.00 samples.], tot_loss[discriminator_loss=2.359, discriminator_real_loss=1.206, discriminator_fake_loss=1.153, generator_loss=30.52, generator_mel_loss=19, generator_kl_loss=1.951, generator_dur_loss=1.46, generator_adv_loss=2.486, generator_feat_match_loss=5.622, over 1674.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:40:03,201 INFO [train.py:845] (2/4) Start epoch 992
2024-02-24 19:43:06,842 INFO [train.py:471] (2/4) Epoch 992, batch 33, global_batch_idx: 36700, batch size: 76, loss[discriminator_loss=2.342, discriminator_real_loss=1.163, discriminator_fake_loss=1.179, generator_loss=30.24, generator_mel_loss=18.7, generator_kl_loss=1.997, generator_dur_loss=1.479, generator_adv_loss=2.348, generator_feat_match_loss=5.723, over 76.00 samples.], tot_loss[discriminator_loss=2.321, discriminator_real_loss=1.171, discriminator_fake_loss=1.15, generator_loss=30.71, generator_mel_loss=18.9, generator_kl_loss=1.972, generator_dur_loss=1.465, generator_adv_loss=2.544, generator_feat_match_loss=5.83, over 2429.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:43:27,673 INFO [train.py:845] (2/4) Start epoch 993
2024-02-24 19:46:57,245 INFO [train.py:845] (2/4) Start epoch 994
2024-02-24 19:48:00,379 INFO [train.py:471] (2/4) Epoch 994, batch 9, global_batch_idx: 36750, batch size: 59, loss[discriminator_loss=2.242, discriminator_real_loss=1.091, discriminator_fake_loss=1.15, generator_loss=30.72, generator_mel_loss=18.59, generator_kl_loss=2.025, generator_dur_loss=1.465, generator_adv_loss=2.734, generator_feat_match_loss=5.898, over 59.00 samples.], tot_loss[discriminator_loss=2.305, discriminator_real_loss=1.169, discriminator_fake_loss=1.136, generator_loss=30.82, generator_mel_loss=18.95, generator_kl_loss=1.967, generator_dur_loss=1.467, generator_adv_loss=2.553, generator_feat_match_loss=5.888, over 732.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 8.0
2024-02-24 19:50:22,737 INFO [train.py:845] (2/4) Start epoch 995
2024-02-24 19:52:40,585 INFO [train.py:471] (2/4) Epoch 995, batch 22, global_batch_idx: 36800, batch size: 63, loss[discriminator_loss=2.326, discriminator_real_loss=1.213, discriminator_fake_loss=1.113, generator_loss=31.07, generator_mel_loss=19.09, generator_kl_loss=1.97, generator_dur_loss=1.456, generator_adv_loss=2.59, generator_feat_match_loss=5.961, over 63.00 samples.], tot_loss[discriminator_loss=2.315, discriminator_real_loss=1.165, discriminator_fake_loss=1.151, generator_loss=30.72, generator_mel_loss=18.96, generator_kl_loss=1.979, generator_dur_loss=1.461, generator_adv_loss=2.535, generator_feat_match_loss=5.784, over 1751.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 16.0
2024-02-24 19:52:40,586 INFO [train.py:525] (2/4) Computing validation loss
2024-02-24 19:52:48,225 INFO [train.py:534] (2/4) Epoch 995, validation: discriminator_loss=2.353, discriminator_real_loss=1.165, discriminator_fake_loss=1.188, generator_loss=31.28, generator_mel_loss=19.39, generator_kl_loss=2.096, generator_dur_loss=1.463, generator_adv_loss=2.517, generator_feat_match_loss=5.814, over 100.00 samples.
2024-02-24 19:52:48,227 INFO [train.py:535] (2/4) Maximum memory allocated so far is 28240MB
2024-02-24 19:53:58,519 INFO [train.py:845] (2/4) Start epoch 996
2024-02-24 19:57:25,934 INFO [train.py:471] (2/4) Epoch 996, batch 35, global_batch_idx: 36850, batch size: 60, loss[discriminator_loss=2.348, discriminator_real_loss=1.107, discriminator_fake_loss=1.239, generator_loss=30.95, generator_mel_loss=19.3, generator_kl_loss=1.978, generator_dur_loss=1.462, generator_adv_loss=2.527, generator_feat_match_loss=5.688, over 60.00 samples.], tot_loss[discriminator_loss=2.339, discriminator_real_loss=1.173, discriminator_fake_loss=1.166, generator_loss=30.68, generator_mel_loss=18.9, generator_kl_loss=1.983, generator_dur_loss=1.461, generator_adv_loss=2.545, generator_feat_match_loss=5.794, over 2579.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 16.0
2024-02-24 19:57:31,454 INFO [train.py:845] (2/4) Start epoch 997
2024-02-24 20:01:02,188 INFO [train.py:845] (2/4) Start epoch 998
2024-02-24 20:02:20,962 INFO [train.py:471] (2/4) Epoch 998, batch 11, global_batch_idx: 36900, batch size: 85, loss[discriminator_loss=2.238, discriminator_real_loss=1.101, discriminator_fake_loss=1.139, generator_loss=30.31, generator_mel_loss=18.53, generator_kl_loss=1.922, generator_dur_loss=1.455, generator_adv_loss=2.492, generator_feat_match_loss=5.914, over 85.00 samples.], tot_loss[discriminator_loss=2.35, discriminator_real_loss=1.186, discriminator_fake_loss=1.164, generator_loss=30.64, generator_mel_loss=18.86, generator_kl_loss=1.948, generator_dur_loss=1.46, generator_adv_loss=2.542, generator_feat_match_loss=5.827, over 821.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 16.0
2024-02-24 20:04:35,581 INFO [train.py:845] (2/4) Start epoch 999
2024-02-24 20:06:56,218 INFO [train.py:471] (2/4) Epoch 999, batch 24, global_batch_idx: 36950, batch size: 52, loss[discriminator_loss=2.316, discriminator_real_loss=1.213, discriminator_fake_loss=1.103, generator_loss=30.16, generator_mel_loss=18.55, generator_kl_loss=1.925, generator_dur_loss=1.475, generator_adv_loss=2.484, generator_feat_match_loss=5.734, over 52.00 samples.], tot_loss[discriminator_loss=2.346, discriminator_real_loss=1.188, discriminator_fake_loss=1.159, generator_loss=30.58, generator_mel_loss=18.84, generator_kl_loss=1.962, generator_dur_loss=1.463, generator_adv_loss=2.555, generator_feat_match_loss=5.753, over 1782.00 samples.], cur_lr_g: 1.77e-04, cur_lr_d: 1.77e-04, grad_scale: 16.0
2024-02-24 20:08:03,052 INFO [train.py:845] (2/4) Start epoch 1000
2024-02-24 20:11:32,951 INFO [train.py:902] (2/4) Done!