[dev-3.x][Fix] Fix duplicate param error in DAB-DETR

[nijkah]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Using torch==1.9.1, mmcv==2.0.0rc2 and mmengine==0.5.0,
When running python3 tools/train.py configs/dab_detr/dab-detr_r50_8xb2-50e_coco.py

File "tools/train.py", line 130, in <module>
    main()
  File "tools/train.py", line 126, in main
    runner.train()
  File "/usr/local/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1661, in train
    self.optim_wrapper = self.build_optim_wrapper(self.optim_wrapper)
  File "/usr/local/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1077, in build_optim_wrapper
    return build_optim_wrapper(self.model, optim_wrapper)
  File "/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/builder.py", line 69, in build_optim_wrapper
    optim_wrapper = optim_wrapper_constructor(model)
  File "/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py", line 301, in __call__
    optimizer = OPTIMIZERS.build(optimizer_cfg)
  File "/usr/local/lib/python3.8/site-packages/mmengine/registry/registry.py", line 521, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/usr/local/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 135, in build_from_cfg
    raise type(e)(
ValueError: class `AdamW` in torch/optim/adamw.py: some parameters appear in more than one parameter group

Modification

Added bypass_duplicate=True to paramwise_cfg as in

mmdetection/configs/resnet_strikes_back/faster-rcnn_r50-rsb-pre_fpn_1x_coco.py

Line 15 in 130a620

paramwise_cfg=dict(norm_decay_mult=0., bypass_duplicate=True))

Printed log

/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: encoder.layers.0.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: encoder.layers.1.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: encoder.layers.2.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: encoder.layers.3.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: encoder.layers.4.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: encoder.layers.5.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: decoder.layers.0.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: decoder.layers.1.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: decoder.layers.2.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: decoder.layers.3.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: decoder.layers.4.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '
/usr/local/lib/python3.8/site-packages/mmengine/optim/optimizer/default_constructor.py:211: UserWarning: decoder.layers.5.ffn.layers.0.1 is duplicate. It is skipped since bypass_duplicate=True
  warnings.warn(f'{prefix} is duplicate. It is skipped since '

[ZwwWayne]

The branch of refactor-detr has been merged into dev-3.x, so we can directly merge this PR into dev-3.x

[LYMDLUT]

We are very grateful for your attention and suggestions, and we also found this phenomenon in the process of replicating the algorithm. This is due to the parameters in the prelu of transformer's FFN and the fact that the FFN is integrated into the mmcv. We have submitted pr to mmcv to solve this problem, we hope you can update mmcv and try it. If there is still a problem, please contact us again, we will continue to follow up.

我们非常感谢您的关注和建议，在我们对算法的复现过程中，同样发现了这种现象。这是因为transformer的FFN的prelu中的参数造成的，并且FFN集成在mmcv中。我们已经向mmcv提交pr解决了这个问题，希望您可以更新mmcv尝试一下，如还存在问题，麻烦您再次联系我们，我们会持续跟进。

[KeiChiTse]

Hi! Thanks for your PR and proposed modification, please refer to #PR 2444 in mmcv and upgrade mmcv, mmengine version accordingly!

[nijkah]

@LYMDLUT @KeiChiTse Thanks for the context!
To support DAB-DETR in mmdetection==3.x normally, should we limit the required minimum version of mmcv?
Or is there any good way to give a good error message to users to upgrade mmcv?

[KeiChiTse]

@LYMDLUT @KeiChiTse Thanks for the context! To support DAB-DETR in mmdetection==3.x normally, should we limit the required minimum version of mmcv? Or is there any good way to give a good error message to users to upgrade mmcv?

Hi! Thanks for your reply and concern. Indeed, the required minimum version of mmcv must be 2.0.0rc4. We will improve this very very soon! Thanks again!

[nijkah]

@KeiChiTse Then, I'll close this PR and wait for the bump PR. Thank you so much 😄