NVIDIA says Snapshot restores replicas with nearly 1 TB checkpoints in under 43 seconds
In NVIDIA's own B200 tests a restore beats a cold start by a wide margin, but the timings leave out container image pulls and need the same GPU model and driver.
Tag SGLang
Qwen 3.8 27B Needs Two RTX 5090s for Full Context
Weights that fit in 32GB still left the RTX 5090 waiting half an hour for a first token, and the fix was a different inference engine rather than a different card.