update readme
Browse files
README.md
CHANGED
@@ -92,6 +92,39 @@ pipeline.load_lora(lora_path, scale=1.0)
|
|
92 |
- Make BF16 _not_ clamp, which improves quality and isn't needed because bf16 is the expected dtype for flux. **I would now recommend always using `"flow_dtype": "bfloat16"` in the config**, though it will slow things down on consumer gpus- but not by much at all since most of the compute still happens via fp8.
|
93 |
- Allow for the T5 Model to be run without any quantization, by specifying `"text_enc_quantization_dtype": "bfloat16"` in the config - or also `"float16"`, though not recommended since t5 deviates a bit when running with float16. I noticed that even with qint8/qfloat8 there is a bit of deviation from bf16 text encoder outputs- so for those who want more accurate / expected text encoder outputs, you can use this option.
|
94 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
95 |
## Installation
|
96 |
|
97 |
This repo _requires_ at least pytorch with cuda=12.4 and an ADA gpu with fp8 support, otherwise `torch._scaled_mm` will throw a CUDA error saying it's not supported. To install with conda/mamba:
|
|
|
92 |
- Make BF16 _not_ clamp, which improves quality and isn't needed because bf16 is the expected dtype for flux. **I would now recommend always using `"flow_dtype": "bfloat16"` in the config**, though it will slow things down on consumer gpus- but not by much at all since most of the compute still happens via fp8.
|
93 |
- Allow for the T5 Model to be run without any quantization, by specifying `"text_enc_quantization_dtype": "bfloat16"` in the config - or also `"float16"`, though not recommended since t5 deviates a bit when running with float16. I noticed that even with qint8/qfloat8 there is a bit of deviation from bf16 text encoder outputs- so for those who want more accurate / expected text encoder outputs, you can use this option.
|
94 |
|
95 |
+
### Updates 10/3/24
|
96 |
+
|
97 |
+
- #### Adding configurable clip model path
|
98 |
+
Now you can specify the clip model's path in the config, using the `clip_path` parameter in a config file.
|
99 |
+
- #### Improved lora loading
|
100 |
+
I believe I have fixed the lora loading bug that was causing the lora to not apply properly, or when not all of the linear weights in the q/k/v/o had loras attached (it wouldn't be able to apply if only some of them did).
|
101 |
+
- #### Lora loading via api endpoint
|
102 |
+
|
103 |
+
You can now post to the `/lora` endpoint with a json file containing a `scale`, `path`, `name`, and `action` parameters.
|
104 |
+
|
105 |
+
The `path` should be the path to the lora safetensors file either absolute or relative to the root of this repo.
|
106 |
+
|
107 |
+
The `name` is an optional parameter, mainly just for checking purposes to see if the correct lora was being loaded, it's used as an identifier to check whether it's already been loaded or which lora to unload if `action` is `unload` (you can also use the exact same path which was loaded previously to unload the same lora).
|
108 |
+
|
109 |
+
The `action` should be either `load` or `unload`, to load or unload the lora.
|
110 |
+
|
111 |
+
The `scale` should be a float, which is the scale of the lora.
|
112 |
+
|
113 |
+
e.g.
|
114 |
+
|
115 |
+
```json
|
116 |
+
{
|
117 |
+
<!-- If you have a lora directory like 'fluxloras' in the root of this repo -->
|
118 |
+
"path": "./fluxloras/loras/aidmaImageUpgrader-FLUX-V0.2.safetensors",
|
119 |
+
<!-- name is optional -->
|
120 |
+
"name": "imgupgrade",
|
121 |
+
<!-- action (load or unload) is required -->
|
122 |
+
"action": "load",
|
123 |
+
<!-- lora scale to use -->
|
124 |
+
"scale": 0.6
|
125 |
+
}
|
126 |
+
```
|
127 |
+
|
128 |
## Installation
|
129 |
|
130 |
This repo _requires_ at least pytorch with cuda=12.4 and an ADA gpu with fp8 support, otherwise `torch._scaled_mm` will throw a CUDA error saying it's not supported. To install with conda/mamba:
|