GPU Compatibility

Which GPUs MemScale runs on, and what to expect on each class of hardware.

Compatibility matrix

Reading the status column

• ✅ Full support — every MemScale technique works, including BF16 mixed precision where the table notes “BF16 native.”
• ⚠️ Partial / Limited / Basic — MemScale runs, but some techniques are reduced. The most common limitation is precision: cards without BF16 hardware fall back to FP16, and the oldest cards have no 16-bit acceleration at all — there, rely on gradient checkpointing and CPU offloading instead of mixed precision.

Precision support, briefly

• BF16 — preferred. Native on RTX 30xx, RTX 40xx, A100, H100.
• FP16 — fallback on RTX 20xx and V100. Works, but has a narrower dynamic range than BF16.
• No 16-bit acceleration — GTX 16xx/10xx. Use checkpointing and offloading; leave use_mixed_precision=False.

See Techniques for how precision interacts with the other optimizations.

Campus / workstation recommendation

For a single fixed training GPU — a lab machine, a campus card, or a personal workstation — the sweet spot is an 8–12 GB GPU, RTX 30xx or newer. That gives native BF16, the full technique stack, and enough VRAM for MemScale to fit meaningfully larger models than the card could handle unoptimized.

CPU-only hosts

MemScale imports and runs without a GPU — useful for inspecting plans or running tests — but the memory optimizations only take effect when a CUDA device is present. With auto_policy=True on a CPU-only host, Stage 1 of the ML policy is skipped, since there is no VRAM pressure to reason about.