Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[hybrid performance] Grad fuse for gradient merge under pipeline mode #35004

Merged
merged 32 commits into from
Aug 20, 2021
Merged

[hybrid performance] Grad fuse for gradient merge under pipeline mode #35004

merged 32 commits into from
Aug 20, 2021

Commits on Aug 19, 2021

  1. fuse grad startup

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    037d647 View commit details
    Browse the repository at this point in the history

  2. remove shape, move fused_merged_grad to startup

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    c9c9e84 View commit details
    Browse the repository at this point in the history

  3. remove merged grad back to main

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    2562a29 View commit details
    Browse the repository at this point in the history

  4. rewrite the dtype

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    b497b1e View commit details
    Browse the repository at this point in the history

  5. add fp32 cast back

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    8553cd9 View commit details
    Browse the repository at this point in the history

  6. create @Grad@MERGED vars

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    2899fae View commit details
    Browse the repository at this point in the history

  7. resource the dtype

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    1e89e78 View commit details
    Browse the repository at this point in the history

  8. add shape info for MERGED var

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    4faee51 View commit details
    Browse the repository at this point in the history

  9. update loop condition

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    3b64dc1 View commit details
    Browse the repository at this point in the history

  10. rewrite cast logic

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    1b9b657 View commit details
    Browse the repository at this point in the history

  11. remove cast_fp16 in grad's name

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    22bfde7 View commit details
    Browse the repository at this point in the history

  12. update grad name generate method

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    72a3120 View commit details
    Browse the repository at this point in the history

  13. copy data for colaesce

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    9050872 View commit details
    Browse the repository at this point in the history

  14. fix dtype of coalesce op

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    ab60822 View commit details
    Browse the repository at this point in the history

  15. dytpe supports for colaesce op

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    4a26c7f View commit details
    Browse the repository at this point in the history

  16. no fp16 allreduce supports

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    d7f0da3 View commit details
    Browse the repository at this point in the history

  17. update dytpe value

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    ff62965 View commit details
    Browse the repository at this point in the history

  18. add size of dtype

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    b112278 View commit details
    Browse the repository at this point in the history

  19. remove startup param

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    3549d31 View commit details
    Browse the repository at this point in the history

  20. rename the dist strategy flag

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    447a193 View commit details
    Browse the repository at this point in the history

  21. rename the size_of_dtype attr for coalesce op

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    d2734bc View commit details
    Browse the repository at this point in the history

  22. update some comment

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    00b7636 View commit details
    Browse the repository at this point in the history

  23. add todo, test=notest

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    e982959 View commit details
    Browse the repository at this point in the history

  24. support NPU set constant for coalesce op, test=notest

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    f120ed7 View commit details
    Browse the repository at this point in the history

  25. add test for optimizer pass, test=notest

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    91f05de View commit details
    Browse the repository at this point in the history

  26. update NPU fill constant, test=notest

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    1257a61 View commit details
    Browse the repository at this point in the history

  27. gpu test for coalesce, test=notest

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    5b02696 View commit details
    Browse the repository at this point in the history

  28. npu test for coalesce

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    98fa317 View commit details
    Browse the repository at this point in the history

  29. remove useless todo

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    bb61c9d View commit details
    Browse the repository at this point in the history

  30. fix bug if user_defined_strategy is None

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    9316944 View commit details
    Browse the repository at this point in the history

  31. tmp solution for fuse_grad_merge + optimize_cast

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    Configuration menu
    Copy the full SHA
    f68967a View commit details
    Browse the repository at this point in the history

  32. add more comment, rename var name

    @FeixLiu
    FeixLiu committed Aug 19, 2021
    1 Configuration menu
    Copy the full SHA
    c87cb9d View commit details
    Browse the repository at this point in the history