NVIDIA H100 80GB HBM3 · median of 10 runs after 3 warm-ups · measured 2026-09-28T20:26:01+00:00
Linnet 0.1.0, PyTorch 2.14.0, JAX 0.11.2, transformers 5.17.0, diffusers 0.40.0, ONNX Runtime 1.30.0, Python 3.12.3, driver 580.126.09
Linnet against the stack it replaces
Each side in its fastest configuration, on the same GPU and checkpoint. A speed-up is how many times the other's speed; an export is measured against the original checkpoint in the same engine.
- Linnet, CUDA graphs34.7 msdiffusers, compiled35.0 ms
Each row also carries its distance from reference: the largest absolute difference between its output and diffusers (eager)'s on the same input. bf16 outputs differ by rounding (about 0.1 on logits near 16), so a small number is expected; it is there so that a fast wrong answer cannot look like a win.
Denoising step
ms, lower is better; the best bar is red.
Load time
s, lower is better; the best bar is red.
Peak GPU memory
What the driver reports the process holding at its peak, in GiB; lower is better; the best bar is red.
Every number
| Method | Denoising step | Load time | Peak GPU memory | Against the stack it replaces | Distance from reference | Notes |
|---|---|---|---|---|---|---|
| PyTorch | ||||||
| diffusers | 48.8 ms | 31.1 s | 6.29 GiB | 0 (the reference) | one step, 128x128 latent (1024-pixel image), batch 2, 77-token context, bf16 | |
| diffusers, compiled | 35.0 ms | 8.13 s | 6.30 GiB | 0.0313 | one step, 128x128 latent (1024-pixel image), batch 2, 77-token context, bf16 | |
| Linnet, generated PyTorch | 56.3 ms | 17.0 s | 6.50 GiB | 0.62× diffusers, compiled | 0.0156 | one step, 128x128 latent (1024-pixel image), batch 2, 77-token context, bf16 |
| Linnet, inductor | 35.7 ms | 17.2 s | 6.28 GiB | 0.98× diffusers, compiled | 0.0313 | one step, 128x128 latent (1024-pixel image), batch 2, 77-token context, bf16 |
| Linnet, CUDA graphs | 34.7 ms | 19.8 s | 6.42 GiB | 1.01× diffusers, compiled | 0.0313 | one step, 128x128 latent (1024-pixel image), batch 2, 77-token context, bf16 |
| JAX | ||||||
| Linnet, XLA (StableHLO) | 53.7 ms | 22.2 s | 6.31 GiB | 0.0313 | one step, 128x128 latent (1024-pixel image), batch 2, 77-token context, bf16; peak memory is JAX's allocator peak; the driver shows its pool, which grows in whole regions | |
| Linnet, XLA (generated JAX) | 39.3 ms | 25.0 s | 5.24 GiB | 0.0313 | one step, 128x128 latent (1024-pixel image), batch 2, 77-token context, bf16; peak memory is JAX's allocator peak; the driver shows its pool, which grows in whole regions | |
| ONNX Runtime and TensorRT | ||||||
| Linnet, ONNX Runtime f32 | 141 ms | 0.30 s | 24.3 GiB | 0.0303 | f32; ONNX Runtime's CUDA execution provider; first calls build the sessions | |
| Linnet, ONNX Runtime f16 | 90.8 ms | 0.29 s | 14.1 GiB | 0.0313 | f16; ONNX Runtime's CUDA execution provider; first calls build the sessions | |
| Linnet, ONNX Runtime bf16 | 100 ms | 0.29 s | 14.6 GiB | 0.0313 | bf16; ONNX Runtime's CUDA execution provider; first calls build the sessions | |
| Linnet, TensorRT f32 | 141 ms | 0.30 s | 24.3 GiB | 0.0303 | f32; ONNX Runtime's TensorRT execution provider; first calls build the sessions | |
| Linnet, TensorRT f16 | 91.4 ms | 0.29 s | 14.1 GiB | 0.0313 | f16; ONNX Runtime's TensorRT execution provider; first calls build the sessions | |
| Linnet, TensorRT bf16 | 100 ms | 0.29 s | 14.6 GiB | 0.0313 | bf16; ONNX Runtime's TensorRT execution provider; first calls build the sessions | |