File size: 2,630 Bytes
cdfff48 43f7a37 cdfff48 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
---
license: apache-2.0
tags:
- image-segmentation
- vision
datasets:
- coco
---
# DETRs with Collaborative Hybrid Assignments Training
## Introduction
In this paper, we present a novel collaborative hybrid assignments training scheme, namely Co-DETR, to learn more efficient and effective DETR-based detectors from versatile label assignment manners.
1. **Encoder optimization**: The proposed training scheme can easily enhance the encoder's learning ability in end-to-end detectors by training multiple parallel auxiliary heads supervised by one-to-many label assignments.
2. **Decoder optimization**: We conduct extra customized positive queries by extracting the positive coordinates from these auxiliary heads to improve attention learning of the decoder.
3. **State-of-the-art performance**: Co-DETR with ViT-Large (304M parameters) is **the first model to achieve 66.0 AP on COCO test-dev.**
## Model Zoo
| Model | Backbone | Aug | Dataset | box AP (val) | mask AP (val) | box AP (test-dev) | mask AP (test-dev) |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Co-DETR | ViT-L | DETR | COCO | 65.3 | 56.2 | - | - |
| Co-DETR (+TTA) | ViT-L | DETR | COCO | 65.8 | 56.6 | 66.0 | 57.1 |
## How to use
We implement Co-DETR using [MMDetection V2.25.3](https://github.com/open-mmlab/mmdetection/releases/tag/v2.25.3) and [MMCV V1.5.0](https://github.com/open-mmlab/mmcv/releases/tag/v1.5.0). Please refer to our [github repo](https://github.com/Sense-X/Co-DETR/tree/main) for more details.
### Training
Train Co-Deformable-DETR + ResNet-50 with 8 GPUs:
```shell
sh tools/dist_train.sh projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py 8 path_to_exp
```
Train using slurm:
```shell
sh tools/slurm_train.sh partition job_name projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py path_to_exp
```
### Testing
Test Co-Deformable-DETR + ResNet-50 with 8 GPUs, and evaluate:
```shell
sh tools/dist_test.sh projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py path_to_checkpoint 8 --eval bbox
```
Test using slurm:
```shell
sh tools/slurm_test.sh partition job_name projects/configs/co_deformable_detr/co_deformable_detr_r50_1x_coco.py path_to_checkpoint --eval bbox
```
## Cite Co-DETR
If you find this repository useful, please use the following BibTeX entry for citation.
```latex
@inproceedings{zong2023detrs,
title={Detrs with collaborative hybrid assignments training},
author={Zong, Zhuofan and Song, Guanglu and Liu, Yu},
booktitle={Proceedings of the IEEE/CVF international conference on computer vision},
pages={6748--6758},
year={2023}
}
``` |