Baltor Get started

Hardware

Can I run it?

Choose your GPU, your Apple or other unified memory, or your computer's memory alone, and see which models and quantizations fit, at which context length. The calculation runs in this page: nothing you enter leaves it.

Check your hardware

What runs the model

Models that fit on the device come first, then models that also need computer memory; within each, the largest models come first, by parameter count, then by name. For each model the check picks the largest quantization that fits. Payment never changes the results.

On GeForce RTX 5090, 32 GB, 751 models fit at 8,192 tokens or more. 383 more have no KV cache numbers in any source, so they are left out.

ModelQuantizationWeightsLongest context that fitsMemory thenWhereSpeed
Ling-flash-2.0IQ2_XXS25.3 GiB32,76829.0 GiBOn the device238 to 380 tokens a second (estimate)
Solar-Open-100BIQ2_XXS25.3 GiB16,38430.1 GiBOn the device140 to 224 tokens a second (estimate)
Qwen3 Next 80B A3B InstructQ2_K27.2 GiB16,38430.5 GiBOn the device249 to 399 tokens a second (estimate)
Qwen3 Next 80B A3B ThinkingQ2_K26.2 GiB32,76831.0 GiBOn the device174 to 279 tokens a second (estimate)
AliceAI-Foundation-80B-A3B-BaseQ2_K27.0 GiB131,07231.9 GiBOn the device173 to 277 tokens a second (estimate)
Hunyuan A13B InstructQ2_K27.4 GiB16,38431.3 GiBOn the device28 to 45 tokens a second (estimate)
Qwen3 Coder NextQ2_K27.2 GiB16,38430.6 GiBOn the device296 to 473 tokens a second (estimate)
Athene-V2-ChatIQ2_XXS23.7 GiB16,38430.4 GiBOn the device29 to 46 tokens a second (estimate)
Qwen2-72BIQ2_XXS23.7 GiB16,38430.4 GiBOn the device29 to 46 tokens a second (estimate)
Qwen2-72B-InstructIQ2_XXS23.7 GiB16,38430.4 GiBOn the device29 to 46 tokens a second (estimate)
Qwen2.5 72B InstructQ2_K25.5 GiB8,19229.7 GiBOn the device30 to 48 tokens a second (estimate)
Qwen2.5-72BIQ2_XXS23.7 GiB16,38430.4 GiBOn the device29 to 46 tokens a second (estimate)
Apertus-70B-Instruct-2509Q2_K25.4 GiB8,19229.7 GiBOn the device30 to 48 tokens a second (estimate)
Athene-70BQ2_K24.6 GiB8,19228.8 GiBOn the device31 to 49 tokens a second (estimate)
Hermes 3 70B InstructQ2_K24.6 GiB16,38431.3 GiBOn the device28 to 45 tokens a second (estimate)
Higgs-Llama-3-70BQ2_K24.6 GiB8,19228.8 GiBOn the device31 to 49 tokens a second (estimate)
Llama 3.3 70B InstructQ2_K24.6 GiB16,38431.3 GiBOn the device28 to 45 tokens a second (estimate)
Llama-3.1-70BQ2_K24.6 GiB16,38431.3 GiBOn the device28 to 45 tokens a second (estimate)
Llama-3.1-Nemotron-70B-Instruct-HFQ2_K24.6 GiB16,38431.3 GiBOn the device28 to 45 tokens a second (estimate)
R1 Distill Llama 70BQ2_K24.6 GiB16,38431.3 GiBOn the device28 to 45 tokens a second (estimate)
Midnight-Miqu-70B-v1.5Q2_K23.7 GiB16,38430.4 GiBOn the device29 to 46 tokens a second (estimate)
Nous-Hermes-2-Mixtral-8x7B-DPOQ4_K_M26.5 GiB16,38430.3 GiBOn the device90 to 143 tokens a second (estimate)
qwen3.8-flash-coder-85gb-bf16Q4_K_M26.4 GiB131,07231.3 GiBOn the device111 to 178 tokens a second (estimate)
Phi-3.5-MoE-instructQ5_K_M27.7 GiB16,38431.6 GiBOn the device131 to 209 tokens a second (estimate)
falcon-40bQ5_K_M28.5 GiB8,19231.4 GiBOn the device28 to 45 tokens a second (estimate)

How the estimate works

The page picks, for each model, the largest quantization that fits, and the longest context that fits with it. Numbers marked estimate are not measurements. Real memory and speed depend on the runtime, its settings and the other work on the machine.

memory = weights + KV cache + overhead
weights  = the size of the quantized files, or parameters x bits per weight / 8
KV cache = 2 x layers x KV heads x head size x context x bytes per value
overhead = 512 MiB + 5% of the weights (an estimate of runtime buffers)
speed    = 50% to 80% of bandwidth / (active weights + KV cache), an estimate

A graphics card may use all its memory; unified memory keeps about a quarter to a third for the system; computer memory alone keeps a fifth. These shares are estimates you can change by entering your own memory. A model that does not fit on the graphics card can still run with some layers in computer memory, much more slowly.

Where the numbers come from

Memory and bandwidth of each preset come from the vendor page below, read on the day shown. A preset without a bandwidth gives no speed until you enter one. Model sizes and architecture numbers come from the model pages. Browse the models or compare endpoints and runtimes.