terry.yaml

---
job: extension
config:
  name: "terry"
  process:
    - type: 'sd_trainer'
      training_folder: "output"
      device: cuda:0
      network:
        type: "lora"
        linear: 16
        linear_alpha: 16
      save:
        dtype: float16 
        save_every: 500 
        max_step_saves_to_keep: 4 
        push_to_hub: true 
        hf_repo_id: maver1chh/terry
        hf_private: false  
      datasets:
        - folder_path: "/home/user0/PDam/ai-toolkit/data/terry"
          caption_ext: "txt"
          caption_dropout_rate: 0.05
          shuffle_tokens: false
          cache_latents_to_disk: true
          resolution: [ 512, 768, 1024 ]
      train:
        batch_size: 1
        steps: 6000 
        gradient_accumulation_steps: 1
        train_unet: true
        train_text_encoder: false  # probably won't work with flux
        gradient_checkpointing: true  # need the on unless you have a ton of vram
        noise_scheduler: "flowmatch" 
        optimizer: "adamw8bit"
        lr: 1e-4
        # uncomment this to skip the pre training sample
#        skip_first_sample: true
        # uncomment to completely disable sampling
#        disable_sampling: true
        # uncomment to use new vell curved weighting. Experimental but may produce better results
#        linear_timesteps: true

        # ema will smooth out learning, but could slow it down. Recommended to leave on.
        ema_config:
          use_ema: true
          ema_decay: 0.99

        # will probably need this if gpu supports it for flux, other dtypes may not work correctly
        dtype: bf16
      model:
        # huggingface model name or path
        name_or_path: "black-forest-labs/FLUX.1-dev"
        is_flux: true
        quantize: true  # run 8bit mixed precision
#        low_vram: true  # uncomment this if the GPU is connected to your monitors. It will use less vram to quantize, but is slower.
      sample:
        sampler: "flowmatch" # must match train.noise_scheduler
        sample_every: 250 # sample every this many steps
        width: 1024
        height: 1024
        prompts:
          # you can add [trigger] to the prompts here and it will be replaced with the trigger word
#          - "[trigger] holding a sign that says 'I LOVE PROMPTS!'"\
          - "Spider-Man as an astronaut traveling to the universe, wearing a futuristic white and red space suit with the signature spider logo on his chest. The suit is highly detailed, with glowing blue lights and web-like textures across the helmet visor and armor plates. Spider-Man is floating weightlessly in deep space, surrounded by vibrant nebulas, distant stars, and glowing planets in the background. His pose is dynamic and heroic, one hand reaching out as if he’s about to shoot a web in zero gravity. The helmet reflects the cosmic scenery, and faint spider-web patterns glimmer in the cosmic light. The atmosphere is awe-inspiring, cinematic, and full of sci-fi realism, blending Spider-Man's iconic superhero aesthetics with the wonders of space."
          - "A superhero sitting at a wooden table in a cozy sushi restaurant, eating sushi with chopsticks. The superhero wears a vibrant, detailed costume with a cape and mask, reflecting their iconic persona. The scene is illustrated in the style of Vincent van Gogh’s paintings, featuring swirling brush strokes, rich colors, and textured patterns reminiscent of Starry Night or Café Terrace at Night. The sushi rolls are meticulously painted with vibrant reds, greens, and whites, arranged beautifully on a wooden platter. The background glows warmly, with the restaurant interior full of swirling lights, lanterns, and dreamy textures. The superhero's pose is relaxed and joyful, appreciating the sushi in this unique, painterly world. The overall atmosphere is whimsical, colorful, and artistic."
        neg: ""  # not used on flux
        seed: 42
        walk_seed: true
        guidance_scale: 4
        sample_steps: 20
# you can add any additional meta info here. [name] is replaced with config name at top
meta:
  name: "[name]"
  version: '1.0'