Integration with PyTorch Lightning

[schopra8]

What would you like to be added: Integrate MS-AMP with PyTorch Lightning

Why is this needed: MS-AMP shows huge gains in throughput when training in FP8. That's very exciting. Adoption and experimentation would easier, if it could be used in common libraries like PyTorch Lightning. The DeepSpeed folks wrote an integration into PyTorch Lightning a little while back -- so as similar collaboration would be awesome to see.

Without this feature, how does current msamp work：You can use DeepSpeed for DDP training.

Components that may involve changes: Not sure

Brief description of your proposal if any: NA