README.md

---
license: cc-by-nc-sa-4.0
---

**Restore missing RGB channels**

Restore a missing channel of a RGB image by using ControlNet to guide image generation of Stable Diffusion to infer missing channel from the other two channels.
* See accompanying discussion at [github.com - Channels RGB](https://github.com/lllyasviel/ControlNet/discussions/567) with detailed report and evaluations.
* To restore images with missing channels you can use [this space](https://huggingface.co/spaces/GeroldMeisinger/channels).
* For evaluation images see the corresponding .zip's at "files".
* To run your own evaluations you can use [this script at gitlab.com](https://gitlab.com/-/snippets/3602096).

# Training

```  
accelerate launch train_controlnet.py \
  --pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5" \
  --train_batch_size=4 \
  --gradient_accumulation_steps=8 \
  --proportion_empty_prompts=0.5
  --mixed_precision="fp16" \
  --learning_rate=1e-5 \
  --enable_xformers_memory_efficient_attention \
  --use_8bit_adam \
  --set_grads_to_none \
  --seed=0 \
  --num_train_epochs=2
```

# Image dataset

* laion2B-en aesthetics>=6.5 dataset
* --min_image_size 512 --max_aspect_ratio 2 --resize_mode="center_crop" --image_size 512
* Cleaned with `fastdup` default settings
* Data augmented with right-left flipped images
* Resulting in 214244 images
* Set whole channel to 0 by alternating between R-G-B channels