Custom H100 GPUs

As with custom A100, NVIDIA sold custom H100 GPUs to at least Meta with 95 GiB (96 GiB?) of HBM2e:¹

Quote

The H100 GPUs used for the experiments are non-standard. They have HBM2e and are limited to a lower TDP. The actual peak TFLOPs should be between SXM and NVL, and we don’t know the exact value.

Footnotes

1. [2410.06511] TorchTitan: One-stop PyTorch native solution for production ready LLM pre-training ↩