Standardise number of GPU Used?

[kowo1001]

Hello, I have three questions.

From the paper 32, 64, 128 NVIDIA V100 GPUs, are used.

(1) I'm trying to make spec-vit-base-ultra-checkpoint-1599.pth, spat-vit-base-ultra-checkpoint-1599.pth foundation model.
Specifically, when can I download HyperGlobal-450K dataset?

(2) How many NVIDA A100 80GB do you think you need to make spec-vit-base-ultra-checkpoint-1599.pth, spat-vit-base-ultra-checkpoint-1599.pth?

(3) How many NVIDIA A100 80GB GPUs do you think each of ViT versions(base, large, and huge) should have?

Any help is highly appreciated ;)

Thanks!

[DotWang]

@kowo1001
For (1), please see #2
For (2)(3), the GPUs that we used have 16GB, so you can manually estimate it.