[Features] Add logger for initialization of parameters

[jshilong]

Motivation

I add an initialization log to record the initialization type of all the parameters. This can help user to use the init_cfg in the BaseModule.

Modification

• Add params_init_info to all modules from top-most level module(such as Detector of mmdetection and Classifier of mmclassification), in mmcv/runner/base_module.py. The params_init_info is a dict, each key is a parameter in the model, and the value is a dict with three keys.

  • params_name(str): the name of parameter

  • init_info(str): The initialization information of the parameter, begin with the type of initialization in INITIALIZERS

    such as XavierInit,KaimingInit and follow with the arguments of initialization, such as

    NormalInit: mean=0, std=0.01
    XavierInit: gain=1, distribution=uniform
    PretrainedInit: load from torchvision://resnet50
    

    Sometimes we don't specify the initialization type or the initialized value is exactly equal to the default initialization of PyTorch, and some BaseModule may call the init_weights during __init__ of the model.

     The value is the same before and after calling `init_weights` of RetinaNet 
    

    At last, If you overload the init_weights of BaseModule and do a self-defined initialization for some parameters

    Initialized by user-defined `init_weights`
    

  • tmp_mean_value (obj:torch.FloatTensor): Current mean value of the parameter, the value would be updated after initialization in INITIALIZERS, when we find it is different with the real mean of parameter, It means that the user initializes the parameter with self-defined init_weightsinstead of init_cfg.

BC-breaking

None

The initialization information would only be dumped into the log file.
Below is the part of the log msg of RetinaNet initialization.

2021-07-19 19:43:36,006 - mmdet - INFO - backbone.layer4.1.conv2.weight - PretrainedInit: load from open-mmlab://regnetx_3.2gf
2021-07-19 19:43:36,006 - mmdet - INFO - backbone.layer4.1.bn2.weight - PretrainedInit: load from open-mmlab://regnetx_3.2gf
2021-07-19 19:43:36,006 - mmdet - INFO - backbone.layer4.1.bn2.bias - PretrainedInit: load from open-mmlab://regnetx_3.2gf
2021-07-19 19:43:36,006 - mmdet - INFO - backbone.layer4.1.conv3.weight - PretrainedInit: load from open-mmlab://regnetx_3.2gf
2021-07-19 19:43:36,007 - mmdet - INFO - backbone.layer4.1.bn3.weight - PretrainedInit: load from open-mmlab://regnetx_3.2gf
2021-07-19 19:43:36,007 - mmdet - INFO - backbone.layer4.1.bn3.bias - PretrainedInit: load from open-mmlab://regnetx_3.2gf
2021-07-19 19:43:36,007 - mmdet - INFO - neck.lateral_convs.0.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.lateral_convs.0.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - neck.lateral_convs.1.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.lateral_convs.1.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - neck.lateral_convs.2.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.lateral_convs.2.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.0.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.0.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.1.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.1.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.2.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.2.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.3.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.3.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.4.conv.weight - XavierInit: gain=1, distribution=uniform, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - neck.fpn_convs.4.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.0.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.0.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.1.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.1.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.2.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.2.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.3.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,007 - mmdet - INFO - bbox_head.cls_convs.3.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.0.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.0.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.1.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.1.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.2.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.2.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.3.conv.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.reg_convs.3.conv.bias - The value is the same before and after calling `init_weights` of RetinaNet 
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.retina_cls.weight - NormalInit: mean=0, std=0.01, bias=-4.59511985013459
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.retina_cls.bias - NormalInit: mean=0, std=0.01, bias=-4.59511985013459
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.retina_reg.weight - NormalInit: mean=0, std=0.01, bias=0
2021-07-19 19:43:36,008 - mmdet - INFO - bbox_head.retina_reg.bias - NormalInit: mean=0, std=0.01, bias=0

[codecov]

Codecov Report

Merging #1150 (d283f29) into master (366c628) will increase coverage by 0.27%.
The diff coverage is 65.88%.

@@            Coverage Diff             @@
##           master    #1150      +/-   ##
==========================================
+ Coverage   68.00%   68.28%   +0.27%     
==========================================
  Files         160      160              
  Lines       10443    10590     +147     
  Branches     1902     1935      +33     
==========================================
+ Hits         7102     7231     +129     
- Misses       2968     2977       +9     
- Partials      373      382       +9     

Flag	Coverage Δ
unittests	68.28% <65.88%> (+0.27%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcv/cnn/utils/weight_init.py	85.18% <43.58%> (-6.16%)	⬇️
mmcv/runner/base_module.py	79.48% <84.78%> (+3.81%)	⬆️
mmcv/utils/logging.py	95.23% <0.00%> (-4.77%)	⬇️
mmcv/ops/upfirdn2d.py	14.58% <0.00%> (-0.72%)	⬇️
mmcv/runner/base_runner.py	78.75% <0.00%> (+0.36%)	⬆️
mmcv/onnx/onnx_utils/symbolic_helper.py	40.69% <0.00%> (+1.16%)	⬆️
mmcv/onnx/symbolic.py	53.44% <0.00%> (+5.17%)	⬆️
mmcv/runner/hooks/lr_updater.py	78.50% <0.00%> (+8.41%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 366c628...d283f29. Read the comment docs.

[ZwwWayne]

@MeowZheng & @nbei may take a look.

[MeowZheng]

LGTM

[MeowZheng]

LGTM