[Model] Support Multi-GPU for Transformer model

[lingfanyu]

Description

This PR uses mutli-process to parallelize training of Transformer model. With batch size 128, 4 GPU gives 3.3x speed up. With batch size 4096, 4 GPU give 3.8x speed up.

Checklist

Please feel free to remove inapplicable items for your PR.

• The PR title starts with [$CATEGORY] (such as [Model], [Doc], [Feature]])
• Changes are complete (i.e. I finished coding on this PR)
• Code is well-documented
• To the my best knowledge, examples are either not affected by this change,
  or have been fixed to be compatible with this change

[lingfanyu]

@yzh119 Let's work together on this PR. Currently, there seems to be a bug in this PR. If you try the grad_accum argument to accumulate gradients of multiple batch, the loss drops very slowly.

Close for now. Reopen when we finish.

[yzh119]

torch.distributed hangs at initialization if we do not specify the first gpu_id to 0.

[yzh119]

I have no write access to your repo, so I put the updated version here.
The issue of not converge may result from incorrect schedule settings, I've fixed the problem and now it successfully converges in 4-GPU setting. (But I think in small data like multi30k, shuffle first then divide is a better choice)

[lingfanyu]

@yzh119 just added you as collaborator. Can you move your changes here?

[yzh119]

With sparse_softmax kernel, the problem of not convergence does not appear anymore.

[lingfanyu]

@yzh119 I think I have finished this PR and fixed the synchronization bug. And now the 4 GPU behaves like single GPU.

Can you review again and then we can merge.