README.md

metadata

license: apache-2.0
language:
  - en
tags:
  - magicdrive
  - image-generation
  - controllable-generation
  - autonomous-driving

MagicDrive-t

ArXiv | Web | License | GitHub (video-branch)

This repository contains 60-frame driving-view video generation model checkpoint.

• We train this model by loading from the 16-frame 224x400 checkpoint (we interpolate the temporal positional embeddings from 16 to 60), and train for 80k steps.
• Please check hydra/overrides.yaml for command overrides in the training config.

MagicDrive: Street View Generation with Diverse 3D Geometry Control
Ruiyuan Gao^1*, Kai Chen^2*, Enze Xie^{3^}, Lanqing Hong³, Zhenguo Li³, Dit-Yan Yeung², Qiang Xu^{1^
1}CUHK ²HKUST ³Huawei Noah's Ark Lab
^*Equal Contribution ^{^}Corresponding Authors

Generated video A.

More cars in A on the right and front.

Generated video B

Change B to night.

Remove the car on the left in B.

Generated video C.

Remove the car on the right in C.

Change C to a rainy day.

Change C to a rainy day and remove the car on the right.

For more information, please refer to our GitHub: https://github.com/cure-lab/MagicDrive/tree/video (i.e., the video branch of MagicDrive repo)