yucornetto commited on
Commit
012c48e
·
verified ·
1 Parent(s): 29dcf96

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -82
README.md CHANGED
@@ -1,82 +1,13 @@
1
- # 1D Visual Tokenization and Generation
2
-
3
- This repo hosts the code and models for the following projects:
4
-
5
- - RAR: [Randomized Autoregressive Visual Generation](https://yucornetto.github.io/projects/rar.html)
6
-
7
- - TiTok: [An Image is Worth 32 Tokens for Reconstruction and Generation](https://yucornetto.github.io/projects/titok.html)
8
-
9
-
10
- ## Short Intro on [Randomized Autoregressive Visual Generation](https://arxiv.org/abs/2406.07550) ([README](README_RAR.md))
11
-
12
- RAR is a an autoregressive (AR) image generator with full compatibility to language modeling. It introduces a randomness annealing strategy with permuted objective at no additional cost, which enhances the model's ability to learn bidirectional contexts while leaving the autoregressive framework intact. RAR sets a FID score 1.48, demonstrating state-of-the-art performance on ImageNet-256 benchmark and significantly outperforming prior AR image generators.
13
-
14
- <p>
15
- <img src="assets/rar_overview.png" alt="teaser" width=90% height=90%>
16
- </p>
17
- <p>
18
- <img src="assets/perf_comp.png" alt="teaser" width=90% height=90%>
19
- </p>
20
-
21
- See more details at [README_RAR](README_RAR.md).
22
-
23
- ## Short Intro on [An Image is Worth 32 Tokens for Reconstruction and Generation](https://arxiv.org/abs/2406.07550) ([README](README_TiTok.md))
24
-
25
- We present a compact 1D tokenizer which can represent an image with as few as 32 discrete tokens. As a result, it leads to a substantial speed-up on the sampling process (e.g., **410 × faster** than DiT-XL/2) while obtaining a competitive generation quality.
26
-
27
- <p>
28
- <img src="assets/titok_teaser.png" alt="teaser" width=90% height=90%>
29
- </p>
30
- <p>
31
- <img src="assets/speed_vs_perf.png" alt="teaser" width=90% height=90%>
32
- </p>
33
-
34
- See more details at [README_TiTok](README_TiTok.md).
35
-
36
- ## Updates
37
- - 11/04/2024: We release the [tech report](https://arxiv.org/abs/2411.00776) and code for RAR models.
38
- - 10/16/2024: We update a set of TiTok tokenizer weights trained with an updated single-stage recipe, leading to easier training and better performance. We release the weight of different model size for both VQ and VAE variants TiTok, which we hope could facilitate the research in this area. More details will be available in a tech report later.
39
- - 09/25/2024: TiTok is accepted by NeurIPS 2024.
40
- - 09/11/2024: Release the training codes of generator based on TiTok.
41
- - 08/28/2024: Release the training codes of TiTok.
42
- - 08/09/2024: Better support on loading pretrained weights from huggingface models, thanks for the help from [@NielsRogge](https://github.com/NielsRogge)!
43
- - 07/03/2024: Evaluation scripts for reproducing the results reported in the paper, checkpoints of TiTok-B64 and TiTok-S128 are available.
44
- - 06/21/2024: Demo code and TiTok-L-32 checkpoints release.
45
- - 06/11/2024: The [tech report](https://arxiv.org/abs/2406.07550) of TiTok is available.
46
-
47
-
48
- ## Installation
49
- ```shell
50
- pip3 install -r requirements.txt
51
- ```
52
-
53
- ## Citing
54
- If you use our work in your research, please use the following BibTeX entry.
55
-
56
- ```BibTeX
57
- @article{yu2024randomized,
58
- author = {Qihang Yu and Ju He and Xueqing Deng and Xiaohui Shen and Liang-Chieh Chen},
59
- title = {Randomized Autoregressive Visual Generation},
60
- journal = {arXiv preprint arXiv:2411.00776},
61
- year = {2024}
62
- }
63
- ```
64
-
65
- ```BibTeX
66
- @article{yu2024an,
67
- author = {Qihang Yu and Mark Weber and Xueqing Deng and Xiaohui Shen and Daniel Cremers and Liang-Chieh Chen},
68
- title = {An Image is Worth 32 Tokens for Reconstruction and Generation},
69
- journal = {NeurIPS},
70
- year = {2024}
71
- }
72
- ```
73
-
74
- ## Acknowledgement
75
-
76
- [MaskGIT](https://github.com/google-research/maskgit)
77
-
78
- [Taming-Transformers](https://github.com/CompVis/taming-transformers)
79
-
80
- [Open-MUSE](https://github.com/huggingface/open-muse)
81
-
82
- [MUSE-Pytorch](https://github.com/baaivision/MUSE-Pytorch)
 
1
+ ---
2
+ license: apache-2.0
3
+ title: RAR
4
+ emoji: 🏆
5
+ colorFrom: indigo
6
+ colorTo: pink
7
+ sdk: gradio
8
+ sdk_version: 4.36.0
9
+ app_file: app.py
10
+ pinned: false
11
+ ---
12
+
13
+ Paper:arxiv.org/abs/2411.00776