[Enhancement] Support LayerScale

[okotaku]

Motivation

Support LayerScale from mmcls.
https://github.com/open-mmlab/mmclassification/blob/dev-1.x/mmcls/models/utils/layer_scale.py

Support LayerScale in FFN.
https://github.com/open-mmlab/mmclassification/blob/dev-1.x/mmcls/models/backbones/deit3.py#L18
It is also used in other backbones such as ViT-Adapter.

Related PR

open-mmlab/mmpretrain#1209
open-mmlab/mmdetection#9354

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
• CLA has been signed and all committers have signed the CLA in this PR.

[codecov]

Codecov Report

Base: 66.62% // Head: 66.69% // Increases project coverage by +0.07% 🎉

Coverage data is based on head (51ffb60) compared to base (104aaa8).
Patch coverage: 100.00% of modified lines in pull request are covered.

❗ Current head 51ffb60 differs from pull request most recent head ec1a7cb. Consider uploading reports for the commit ec1a7cb to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##              2.x    #2451      +/-   ##
==========================================
+ Coverage   66.62%   66.69%   +0.07%     
==========================================
  Files         120      120              
  Lines        7799     7816      +17     
  Branches     1067     1072       +5     
==========================================
+ Hits         5196     5213      +17     
  Misses       2436     2436              
  Partials      167      167              

Flag	Coverage Δ
unittests	66.69% <100.00%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcv/cnn/bricks/__init__.py	100.00% <100.00%> (ø)
mmcv/cnn/bricks/scale.py	100.00% <100.00%> (ø)
mmcv/cnn/bricks/transformer.py	84.28% <100.00%> (+0.15%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[HAOCHENYE]

Hi! thanks for your contribution!
It seems

mmcv/mmcv/cnn/bricks/scale.py

Line 6 in ece11bc

class Scale(nn.Module):

is similar to LayerScale, could implement LayerScale there?

[HAOCHENYE]

Hi ! CI seems failed now and we need to fix it 😸

[okotaku]

@HAOCHENYE I have fixed test.

[HAOCHENYE]

LGTM 👍 ! My last comment: Please also update LayerScale in https://github.com/open-mmlab/mmcv/blob/2.x/docs/en/api/cnn.rst and https://github.com/open-mmlab/mmcv/blob/2.x/docs/zh_cn/api/cnn.rst.

[okotaku]

Thank you for your reviews!

[HAOCHENYE]

Approved!!!