Open-weight models on your hardware

Set your hardware once. Every open-weight model is graded on it at the precision that suits it, with the memory it needs, an estimated speed and the longest context that fits. Estimates, not measurements.

My hardware GeForce RTX 4090 24 GB · 1008 GB/s · 64 GB DDR5 dual channel · 89.6 GB/s

Search Precision Context Concurrent requests Minimum speed Architecture Type Maker Sort

Fit	Model	Parameters / active	Precision	Memory	Est. speed	Longest context	Runs as
Runs well	Qwen3.5-35B-A3B Alibaba Qwen	36B / 3B	IQ4_XS	18.8 / 21.6 GB	286–429 tok/s	149K	One GPU
Runs well	Qwen3.6-35B-A3B Alibaba Qwen	36B / 3B	IQ4_XS	18.8 / 21.6 GB	286–429 tok/s	149K	One GPU
Runs well	Qwen2.5-VL-32B-Instruct Alibaba Qwen	33.5B	IQ4_XS	19.4 / 21.6 GB	25–38 tok/s	17K	One GPU
Runs well	Qwen3-VL-32B-Instruct Alibaba Qwen	33.4B	IQ4_XS	19.3 / 21.6 GB	25–38 tok/s	17K	One GPU
Runs well	Qwen3-VL-32B-Thinking Alibaba Qwen	33.4B	IQ4_XS	19.3 / 21.6 GB	25–38 tok/s	17K	One GPU
Runs well	Qwen2.5-32B-Instruct Alibaba Qwen	32.8B	IQ4_XS	19.0 / 21.6 GB	26–39 tok/s	18K	One GPU
Runs well	Qwen2.5-Coder-32B-Instruct Alibaba Qwen	32.8B	IQ4_XS	19.0 / 21.6 GB	26–39 tok/s	18K	One GPU
Runs well	QwQ-32B Alibaba Qwen	32.8B	IQ4_XS	19.0 / 21.6 GB	26–39 tok/s	18K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-32B DeepSeek	32.8B	IQ4_XS	19.0 / 21.6 GB	26–39 tok/s	18K	One GPU
Runs well	Qwen3-32B Alibaba Qwen	32.8B	IQ4_XS	19.0 / 21.6 GB	26–39 tok/s	18K	One GPU
Runs well	GLM-4-32B-0414 Zhipu AI	32.6B	Q4_0	18.4 / 21.6 GB	27–40 tok/s	32K	One GPU
Runs well	Qwen3-Omni-30B-A3B-Captioner Alibaba Qwen	31.7B	Q4_0	18.2 / 21.6 GB	150–225 tok/s	44K	One GPU
Runs well	Gemma 4 31B IT Google	31.3B	Q4_0	19.2 / 21.6 GB	25–38 tok/s	23K	One GPU
Runs well	GLM-4.7-Flash Zhipu AI	31.2B	Q4_K_M	19.0 / 21.6 GB	140–211 tok/s	59K	One GPU
Runs well	Qwen3-VL-30B-A3B-Instruct Alibaba Qwen	31.1B	Q4_K_Mmaker’s file	18.9 / 21.6 GB	161–242 tok/s	37K	One GPU
Runs well	Qwen3-VL-30B-A3B-Thinking Alibaba Qwen	31.1B	Q4_K_Mmaker’s file	18.9 / 21.6 GB	161–242 tok/s	37K	One GPU
Runs well	Qwen3-30B-A3B-Instruct-2507 Alibaba Qwen	30.5B / 3.3B	Q4_K_M	18.9 / 21.6 GB	179–269 tok/s	37K	One GPU
Runs well	Qwen3-30B-A3B-Thinking-2507 Alibaba Qwen	30.5B / 3.3B	Q4_K_M	18.9 / 21.6 GB	179–269 tok/s	37K	One GPU
Runs well	Qwen3-30B-A3B Alibaba Qwen	30.5B / 3.3B	Q4_K_Mmaker’s file	18.9 / 21.6 GB	179–269 tok/s	37K	One GPU
Runs well	Qwen3-Coder-30B-A3B-Instruct Alibaba Qwen	30.5B / 3.3B	Q4_K_M	18.9 / 21.6 GB	179–269 tok/s	37K	One GPU
Runs well	Hy-Memory-30B-A3B Tencent	30.1B / 3B	Q4_K_Mmaker’s file	18.6 / 21.6 GB	192–288 tok/s	40K	One GPU
Runs well	Granite Switch 4.1 30B IBM	30B	IQ4_XS	17.6 / 21.6 GB	28–42 tok/s	23K	One GPU
Runs well	Qwen3.5-27B Alibaba Qwen	27.8B	Q5_0	19.2 / 21.6 GB	26–38 tok/s	46K	One GPU
Runs well	Qwen3.6-27B Alibaba Qwen	27.8B	Q5_0	19.2 / 21.6 GB	26–38 tok/s	46K	One GPU
Runs well	Qwen3.8-27B Alibaba Qwen	27.8B	Q5_0	19.2 / 21.6 GB	26–38 tok/s	46K	One GPU
Runs well	MiniMax-H3-Image-PE MiniMax	27.4B	Q5_0	18.9 / 21.6 GB	26–39 tok/s	51K	One GPU
Runs well	Gemma 4 26B A4B IT Google	25.8B / 3.8B	Q5_K_M	18.5 / 21.6 GB	155–233 tok/s	88K	One GPU
Runs well	Magistral Small 1.2 Mistral AI	24B	Q5_K_Mmaker’s file	17.6 / 21.6 GB	28–42 tok/s	33K	One GPU
Runs well	Devstral Small 1.0 Mistral AI	23.6B	Q5_K_M	17.6 / 21.6 GB	28–42 tok/s	33K	One GPU
Runs well	Devstral Small 1.1 Mistral AI	23.6B	Q5_K_M	17.6 / 21.6 GB	28–42 tok/s	33K	One GPU
Runs well	Magistral Small 1.0 Mistral AI	23.6B	Q5_K_M	17.6 / 21.6 GB	28–42 tok/s	33K	One GPU
Runs well	Magistral Small 1.1 Mistral AI	23.6B	Q5_K_Mmaker’s file	17.6 / 21.6 GB	28–42 tok/s	33K	One GPU
Runs well	ERNIE-4.5-21B-A3B-Thinking Baidu	21.8B / 3B	Q6_K	18.0 / 21.6 GB	172–258 tok/s	74K	One GPU
Runs well	GPT OSS 20B OpenAI	20.9B / 3.6B	MXFP4 (native)maker’s file	13.6 / 21.6 GB	196–294 tok/s	128K	One GPU
Runs well	Qwen2.5-14B-Instruct-1M Alibaba Qwen	14.8B	Q8_0	16.8 / 21.6 GB	29–44 tok/s	33K	One GPU
Runs well	Qwen2.5-14B-Instruct Alibaba Qwen	14.8B	Q8_0maker’s file	16.9 / 21.6 GB	29–44 tok/s	32K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-14B DeepSeek	14.8B	Q8_0	16.8 / 21.6 GB	29–44 tok/s	32K	One GPU
Runs well	Qwen3-14B Alibaba Qwen	14.8B	Q8_0maker’s file	16.6 / 21.6 GB	30–44 tok/s	40K	One GPU
Runs well	Gemma 3 12B IT Google	12.2B	Q8_0	15.5 / 21.6 GB	32–47 tok/s	25K	One GPU
Runs well	Qwen2.5-Omni-7B Alibaba Qwen	10.7B	Q8_0	11.6 / 21.6 GB	42–64 tok/s	32K	One GPU
Runs well	GLM-4.6V-Flash Zhipu AI	10.3B	Q8_0	11.0 / 21.6 GB	45–67 tok/s	125K	One GPU
Runs well	Qwen3.5-9B Alibaba Qwen	9.7B	BF16 (native)maker’s file	19.1 / 21.6 GB	26–39 tok/s	87K	One GPU
Runs well	Qianfan-VL-8B Baidu	8.8B	BF16 (native)maker’s file	18.2 / 21.6 GB	27–40 tok/s	32K	One GPU
Runs well	Granite Switch 4.1 8B IBM	8.8B	BF16 (native)maker’s file	18.5 / 21.6 GB	27–40 tok/s	27K	One GPU
Runs well	Qwen3-VL-8B-Instruct Alibaba Qwen	8.8B	BF16 (native)maker’s file	18.3 / 21.6 GB	27–40 tok/s	31K	One GPU
Runs well	Qwen3-VL-8B-Thinking Alibaba Qwen	8.8B	BF16 (native)maker’s file	18.3 / 21.6 GB	27–40 tok/s	31K	One GPU
Runs well	pplx-embed-v2-context-9b Perplexity	8.4B	Q8_0	9.1 / 21.6 GB	55–82 tok/s	256K	One GPU
Runs well	Qwen2.5-VL-7B-Instruct Alibaba Qwen	8.3B	BF16 (native)maker’s file	16.7 / 21.6 GB	30–44 tok/s	98K	One GPU
Runs well	Qwen3-8B Alibaba Qwen	8.2B	BF16 (native)maker’s file	17.1 / 21.6 GB	29–43 tok/s	39K	One GPU
Runs well	DeepSeek-R1-Distill-Llama-8B DeepSeek	8B	BF16 (native)maker’s file	16.7 / 21.6 GB	29–44 tok/s	47K	One GPU
Runs well	Gemma 4 E4B Google	8B	BF16 (native)maker’s file	15.8 / 21.6 GB	31–47 tok/s	125K	One GPU
Runs well	Ling-3.0-tiny Ant Group	7.9B / 1.3B	BF16 (native)maker’s file	15.6 / 21.6 GB	178–267 tok/s	128K	One GPU
Runs well	Qwen2.5-7B-Instruct-1M Alibaba Qwen	7.6B	BF16 (native)maker’s file	15.3 / 21.6 GB	32–48 tok/s	122K	One GPU
Runs well	Qwen2.5-7B-Instruct Alibaba Qwen	7.6B	BF16 (native)maker’s file	15.3 / 21.6 GB	32–48 tok/s	32K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-7B DeepSeek	7.6B	BF16 (native)maker’s file	15.3 / 21.6 GB	32–48 tok/s	32K	One GPU
Runs well	Mistral-7B-Instruct-v0.1 Mistral AI	7.2B	BF16 (native)maker’s file	15.2 / 21.6 GB	32–49 tok/s	32K	One GPU
Runs well	Gemma 4 E2B Google	5.1B	BF16 (native)maker’s file	10.1 / 21.6 GB	49–73 tok/s	125K	One GPU
Runs well	WeVisOne-VL-4B Tencent	4.8B	BF16 (native)maker’s file	10.6 / 21.6 GB	46–70 tok/s	86K	One GPU
Runs well	Qianfan-OCR Baidu	4.7B	BF16 (native)maker’s file	10.5 / 21.6 GB	47–71 tok/s	31K	One GPU
Runs well	Qwen3.5-4B Alibaba Qwen	4.7B	BF16 (native)maker’s file	9.4 / 21.6 GB	53–79 tok/s	256K	One GPU
Runs well	Qwen3-VL-4B-Instruct Alibaba Qwen	4.4B	BF16 (native)maker’s file	9.9 / 21.6 GB	50–75 tok/s	91K	One GPU
Runs well	Qwen3-4B-Instruct-2507 Alibaba Qwen	4B	BF16 (native)maker’s file	9.1 / 21.6 GB	54–82 tok/s	97K	One GPU
Runs well	Qwen3-4B Alibaba Qwen	4B	BF16 (native)maker’s file	9.1 / 21.6 GB	54–82 tok/s	128K	One GPU
Runs well	DeepSeek-OCR DeepSeek	3.3B	BF16 (native)maker’s file	7.2 / 21.6 GB	181–271 tok/s	8K	One GPU
Runs well	Granite-4.0-H-Micro IBM	3.2B	BF16 (native)maker’s file	6.5 / 21.6 GB	78–117 tok/s	128K	One GPU
Runs well	Qwen3-1.7B Alibaba Qwen	2B	BF16 (native)maker’s file	5.2 / 21.6 GB	101–151 tok/s	40K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-1.5B DeepSeek	1.8B	BF16 (native)maker’s file	4.0 / 21.6 GB	133–200 tok/s	125K	One GPU
Runs well	GLM-OCR Zhipu AI	1.3B	BF16 (native)maker’s file	3.5 / 21.6 GB	158–237 tok/s	128K	One GPU
Runs well	Qwen3-0.6B Alibaba Qwen	0.8B	BF16 (native)maker’s file	2.8 / 21.6 GB	206–309 tok/s	40K	One GPU
Runs well	EmbeddingGemma 2 Google	0.7B	BF16 (native)maker’s file	2.0 / 21.6 GB	320–479 tok/s	256K	One GPU
Runs well	Qwen3-Embedding-0.6B Alibaba Qwen	0.6B	BF16 (native)maker’s file	2.5 / 21.6 GB	236–355 tok/s	32K	One GPU
Runs well	bge-reranker-v2-m3 BAAI	0.6B	FP32 (native)maker’s file	3.4 / 21.6 GB	164–246 tok/s	8K	One GPU
Runs well	bge-m3-unsupervised BAAI	0.6B	FP32 (native)maker’s file	3.4 / 21.6 GB	164–246 tok/s	8K	One GPU
Runs well	bge-large-en-v1.5 BAAI	0.3B	FP32 (native)maker’s file	1.8 / 21.6 GB	362–544 tok/s	512	One GPU
Runs well	bge-reranker-base BAAI	0.3B	FP32 (native)maker’s file	1.6 / 21.6 GB	446–668 tok/s	514	One GPU
Runs well	bge-base-en-v1.5 BAAI	0.1B	FP32 (native)maker’s file	0.9 / 21.6 GB	1103–1655 tok/s	512	One GPU
Runs well	bge-small-en-v1.5 BAAI	0B	FP32 (native)maker’s file	0.6 / 21.6 GB	3527–5290 tok/s	512	One GPU
Needs system memory	Ming-3.0-audio-flash Ant Group	128.2B / 6.5B	Q4_K_M	76.5 GB, 55.3 in system memory	14–22 tok/s	—	32/42 layers in system memory
Needs system memory	Ling-3.0-flash Ant Group	127.5B / 5.1B	Q4_K_Mmaker’s file	75.7 GB, 54.6 in system memory	19–28 tok/s	—	32/42 layers in system memory
Needs system memory	Qwen3.5-122B-A10B Alibaba Qwen	125.1B / 10B	IQ4_XS	65.2 GB, 43.8 in system memory	11–17 tok/s	—	34/48 layers in system memory
Needs system memory	Ling-3.0-flash-VL Ant Group	124.8B / 5.5B	Q4_K_M	74.5 GB, 53.8 in system memory	17–26 tok/s	—	32/42 layers in system memory
Needs system memory	gpt-oss-120b OpenAI	116.8B / 5.1B	MXFP4 (native)maker’s file	64.1 GB, 43.9 in system memory	21–32 tok/s	—	26/36 layers in system memory
Needs system memory	GLM-4.5-Air Zhipu AI	110.5B	IQ4_XS	58.8 GB, 49.3 in system memory	12–18 tok/s	—	Experts in system memory
Needs system memory	GLM-4.5V Zhipu AI	107.7B / 12B	IQ4_XS	57.4 GB, 50.5 in system memory	12–18 tok/s	—	Experts in system memory
Needs system memory	GLM-4.6V Zhipu AI	107.7B	IQ4_XS	57.4 GB, 49.3 in system memory	12–18 tok/s	—	Experts in system memory
Needs system memory	Ling-2.6-flash Ant Group	107.5B / 7.4B	Q4_K_M	64.1 GB, 43.7 in system memory	13–20 tok/s	—	23/32 layers in system memory
Needs system memory	Qwen3-Next-80B-A3B-Instruct Alibaba Qwen	81.3B / 3B	Q6_Kmaker’s file	64.3 GB, 43.2 in system memory	25–38 tok/s	—	34/48 layers in system memory
Needs system memory	Qwen3-Next-80B-A3B-Thinking Alibaba Qwen	81.3B / 3B	Q6_Kmaker’s file	64.3 GB, 43.2 in system memory	25–38 tok/s	—	34/48 layers in system memory
Needs system memory	Qwen3-Coder-Next Alibaba Qwen	79.7B / 3B	FP8maker’s file	78.8 GB, 57.7 in system memory	19–28 tok/s	—	37/48 layers in system memory
Needs system memory	Qwen2.5-VL-72B-Instruct Alibaba Qwen	73.4B	IQ4_XS	40.6 GB, 19.1 in system memory	2–3 tok/s	—	42/80 layers in system memory
Needs system memory	Qwen2-VL-72B-Instruct Alibaba Qwen	73.4B	IQ4_XS	40.6 GB, 19.1 in system memory	2–3 tok/s	—	42/80 layers in system memory
Needs system memory	Qwen2.5-72B-Instruct Alibaba Qwen	72.7B	IQ4_XS	40.3 GB, 18.9 in system memory	2–3 tok/s	—	42/80 layers in system memory
Needs system memory	Qianfan-VL-70B Baidu	71.8B	IQ4_XS	39.8 GB, 18.7 in system memory	2–3 tok/s	—	42/80 layers in system memory
Needs system memory	DeepSeek-R1-Distill-Llama-70B DeepSeek	70.6B	IQ4_XS	39.2 GB, 17.9 in system memory	2–3 tok/s	—	41/80 layers in system memory

Estimates from the makers’ Hugging Face repositories (as of 2026-11-11); filtered views aren’t indexed. How it’s estimated · All hardware