For the GPT-J 6B LLM, Nvidia showed that an H100 enabled with TensorRT-LLM can perform inference two times faster than a regular H100 and eight times faster than the previous-generation A100.

When asked to compare H100 to Intel’s Habana Gaudi2, the deep learning processors that Intel insists have outperformed Nvidia’s A100, Salvator said he was uncertain. “In terms of any sort of ...