Open-weight models that run on GeForce RTX 3080

Memory 10 GB
Bandwidth 760 GB/s
FP16 dense 59.5 TFLOPS
Usable 9.0 GB

Specs: Memory · NVIDIA RTX Blackwell GPU Architecture (whitepaper) · Bandwidth · NVIDIA RTX Blackwell GPU Architecture (whitepaper) · FP16 dense · NVIDIA RTX Blackwell GPU Architecture (whitepaper)

Search Precision Context Concurrent requests Minimum speed Architecture Type Maker Sort

Fit	Model	Parameters / active	Precision	Memory	Est. speed	Longest context	Runs as
Runs well	Qwen2.5-Omni-7B Alibaba Qwen	10.7B	Q5_K_M	8.0 / 9.0 GB	47–70 tok/s	25K	One GPU
Runs well	GLM-4.6V-Flash Zhipu AI	10.3B	Q5_K_M	7.6 / 9.0 GB	50–74 tok/s	43K	One GPU
Runs well	Qwen3.5-9B Alibaba Qwen	9.7B	Q5_K_M	7.1 / 9.0 GB	53–80 tok/s	67K	One GPU
Runs well	Qianfan-VL-8B Baidu	8.8B	Q5_K_M	7.3 / 9.0 GB	52–78 tok/s	21K	One GPU
Runs well	Granite Switch 4.1 8B IBM	8.8B	Q4_K_Mmaker’s file	7.8 / 9.0 GB	49–73 tok/s	15K	One GPU
Runs well	Qwen3-VL-8B-Instruct Alibaba Qwen	8.8B	Q5_K_M	7.4 / 9.0 GB	51–77 tok/s	19K	One GPU
Runs well	Qwen3-VL-8B-Thinking Alibaba Qwen	8.8B	Q5_K_M	7.4 / 9.0 GB	51–77 tok/s	19K	One GPU
Runs well	pplx-embed-v2-context-9b Perplexity	8.4B	Q6_K	7.2 / 9.0 GB	53–80 tok/s	66K	One GPU
Runs well	Qwen2.5-VL-7B-Instruct Alibaba Qwen	8.3B	Q6_K	7.3 / 9.0 GB	52–78 tok/s	39K	One GPU
Runs well	Qwen3-8B Alibaba Qwen	8.2B	Q6_Kmaker’s file	7.9 / 9.0 GB	48–72 tok/s	16K	One GPU
Runs well	DeepSeek-R1-Distill-Llama-8B DeepSeek	8B	Q6_K	7.6 / 9.0 GB	50–74 tok/s	18K	One GPU
Runs well	Gemma 4 E4B Google	8B	FP8	8.1 / 9.0 GB	47–70 tok/s	74K	One GPU
Runs well	Ling-3.0-tiny Ant Group	7.9B / 1.3B	Q6_Kmaker’s file	6.8 / 9.0 GB	293–440 tok/s	93K	One GPU
Runs well	Qwen2.5-7B-Instruct-1M Alibaba Qwen	7.6B	FP8	8.0 / 9.0 GB	47–70 tok/s	25K	One GPU
Runs well	Qwen2.5-7B-Instruct Alibaba Qwen	7.6B	FP8	8.0 / 9.0 GB	47–70 tok/s	26K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-7B DeepSeek	7.6B	FP8	8.0 / 9.0 GB	47–70 tok/s	26K	One GPU
Runs well	Mistral-7B-Instruct-v0.1 Mistral AI	7.2B	Q6_K	7.0 / 9.0 GB	54–81 tok/s	23K	One GPU
Runs well	Gemma 4 E2B Google	5.1B	Q8_0	5.6 / 9.0 GB	69–103 tok/s	125K	One GPU
Runs well	WeVisOne-VL-4B Tencent	4.8B	Q8_0	6.4 / 9.0 GB	60–90 tok/s	26K	One GPU
Runs well	Qianfan-OCR Baidu	4.7B	Q8_0	6.3 / 9.0 GB	61–91 tok/s	27K	One GPU
Runs well	Qwen3.5-4B Alibaba Qwen	4.7B	Q8_0	5.4 / 9.0 GB	73–109 tok/s	124K	One GPU
Runs well	Qwen3-VL-4B-Instruct Alibaba Qwen	4.4B	Q8_0maker’s file	5.6 / 9.0 GB	69–104 tok/s	32K	One GPU
Runs well	Qwen3-4B-Instruct-2507 Alibaba Qwen	4B	Q8_0	5.6 / 9.0 GB	69–104 tok/s	32K	One GPU
Runs well	Qwen3-4B Alibaba Qwen	4B	Q8_0maker’s file	5.6 / 9.0 GB	69–104 tok/s	32K	One GPU
Runs well	DeepSeek-OCR DeepSeek	3.3B	BF16 (native)maker’s file	7.2 / 9.0 GB	136–205 tok/s	8K	One GPU
Runs well	Granite-4.0-H-Micro IBM	3.2B	BF16 (native)maker’s file	6.5 / 9.0 GB	59–88 tok/s	128K	One GPU
Runs well	Qwen3-1.7B Alibaba Qwen	2B	BF16 (native)maker’s file	5.2 / 9.0 GB	76–114 tok/s	40K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-1.5B DeepSeek	1.8B	BF16 (native)maker’s file	4.0 / 9.0 GB	100–150 tok/s	125K	One GPU
Runs well	GLM-OCR Zhipu AI	1.3B	BF16 (native)maker’s file	3.5 / 9.0 GB	119–179 tok/s	96K	One GPU
Runs well	Qwen3-0.6B Alibaba Qwen	0.8B	BF16 (native)maker’s file	2.8 / 9.0 GB	156–233 tok/s	40K	One GPU
Runs well	EmbeddingGemma 2 Google	0.7B	BF16 (native)maker’s file	2.0 / 9.0 GB	241–361 tok/s	256K	One GPU
Runs well	Qwen3-Embedding-0.6B Alibaba Qwen	0.6B	BF16 (native)maker’s file	2.5 / 9.0 GB	178–267 tok/s	32K	One GPU
Runs well	bge-reranker-v2-m3 BAAI	0.6B	FP32 (native)maker’s file	3.4 / 9.0 GB	124–185 tok/s	8K	One GPU
Runs well	bge-m3-unsupervised BAAI	0.6B	FP32 (native)maker’s file	3.4 / 9.0 GB	124–185 tok/s	8K	One GPU
Runs well	bge-large-en-v1.5 BAAI	0.3B	FP32 (native)maker’s file	1.8 / 9.0 GB	273–410 tok/s	512	One GPU
Runs well	bge-reranker-base BAAI	0.3B	FP32 (native)maker’s file	1.6 / 9.0 GB	336–504 tok/s	514	One GPU
Runs well	bge-base-en-v1.5 BAAI	0.1B	FP32 (native)maker’s file	0.9 / 9.0 GB	832–1248 tok/s	512	One GPU
Runs well	bge-small-en-v1.5 BAAI	0B	FP32 (native)maker’s file	0.6 / 9.0 GB	2659–3989 tok/s	512	One GPU
Needs system memory	Qwen3.5-122B-A10B Alibaba Qwen	125.1B / 10B	IQ4_XS	65.2 GB, 56.7 in system memory	9–14 tok/s	—	44/48 layers in system memory
Needs system memory	Ling-3.0-flash-VL Ant Group	124.8B / 5.5B	IQ4_XS	65.2 GB, 57.4 in system memory	16–25 tok/s	—	39/42 layers in system memory
Needs system memory	gpt-oss-120b OpenAI	116.8B / 5.1B	MXFP4 (native)maker’s file	64.1 GB, 55.7 in system memory	17–25 tok/s	—	33/36 layers in system memory
Needs system memory	GLM-4.5-Air Zhipu AI	110.5B	IQ4_XS	58.8 GB, 49.9 in system memory	5–8 tok/s	—	42/46 layers in system memory
Needs system memory	GLM-4.5V Zhipu AI	107.7B / 12B	IQ4_XS	57.4 GB, 50.5 in system memory	12–18 tok/s	—	Experts in system memory
Needs system memory	GLM-4.6V Zhipu AI	107.7B	IQ4_XS	57.4 GB, 49.3 in system memory	12–18 tok/s	—	Experts in system memory
Needs system memory	Ling-2.6-flash Ant Group	107.5B / 7.4B	Q4_0	59.4 GB, 54.1 in system memory	21–32 tok/s	—	Experts in system memory
Needs system memory	Qwen3-Next-80B-A3B-Instruct Alibaba Qwen	81.3B / 3B	Q6_Kmaker’s file	64.3 GB, 55.9 in system memory	20–30 tok/s	—	44/48 layers in system memory
Needs system memory	Qwen3-Next-80B-A3B-Thinking Alibaba Qwen	81.3B / 3B	Q6_Kmaker’s file	64.3 GB, 55.9 in system memory	20–30 tok/s	—	44/48 layers in system memory
Needs system memory	Qwen3-Coder-Next Alibaba Qwen	79.7B / 3B	Q6_Kmaker’s file	64.3 GB, 55.9 in system memory	20–29 tok/s	—	44/48 layers in system memory
Needs system memory	Qwen3.5-35B-A3B Alibaba Qwen	36B / 3B	Q8_0	37.5 GB, 33.7 in system memory	33–49 tok/s	—	Experts in system memory
Needs system memory	Qwen3.6-35B-A3B Alibaba Qwen	36B / 3B	Q8_0	37.5 GB, 33.7 in system memory	33–49 tok/s	—	Experts in system memory
Needs system memory	Qwen2.5-VL-32B-Instruct Alibaba Qwen	33.5B	IQ4_XS	19.4 GB, 10.6 in system memory	4–6 tok/s	—	41/64 layers in system memory
Needs system memory	Qwen3-VL-32B-Instruct Alibaba Qwen	33.4B	IQ4_XS	19.3 GB, 10.6 in system memory	4–6 tok/s	—	41/64 layers in system memory
Needs system memory	Qwen3-VL-32B-Thinking Alibaba Qwen	33.4B	IQ4_XS	19.3 GB, 10.6 in system memory	4–6 tok/s	—	41/64 layers in system memory
Needs system memory	Qwen2.5-32B-Instruct Alibaba Qwen	32.8B	IQ4_XS	19.0 GB, 10.1 in system memory	4–6 tok/s	—	40/64 layers in system memory
Needs system memory	Qwen2.5-Coder-32B-Instruct Alibaba Qwen	32.8B	IQ4_XS	19.0 GB, 10.1 in system memory	4–6 tok/s	—	40/64 layers in system memory
Needs system memory	QwQ-32B Alibaba Qwen	32.8B	IQ4_XS	19.0 GB, 10.1 in system memory	4–6 tok/s	—	40/64 layers in system memory
Needs system memory	DeepSeek-R1-Distill-Qwen-32B DeepSeek	32.8B	IQ4_XS	19.0 GB, 10.1 in system memory	4–6 tok/s	—	40/64 layers in system memory
Needs system memory	Qwen3-32B Alibaba Qwen	32.8B	IQ4_XS	19.0 GB, 10.1 in system memory	4–6 tok/s	—	40/64 layers in system memory
Needs system memory	Qwen3-Omni-30B-A3B-Captioner Alibaba Qwen	31.7B	Q8_0	33.7 GB, 28.7 in system memory	20–30 tok/s	—	Experts in system memory
Needs system memory	Gemma 4 31B IT Google	31.3B	IQ4_XS	18.3 GB, 9.3 in system memory	4–6 tok/s	—	36/60 layers in system memory
Needs system memory	GLM-4.7-Flash Zhipu AI	31.2B	Q8_0	32.9 GB, 27.5 in system memory	20–30 tok/s	—	Experts in system memory
Needs system memory	Qwen3-VL-30B-A3B-Instruct Alibaba Qwen	31.1B	Q8_0maker’s file	32.5 GB, 28.2 in system memory	21–31 tok/s	—	Experts in system memory
Needs system memory	Qwen3-VL-30B-A3B-Thinking Alibaba Qwen	31.1B	Q8_0maker’s file	32.5 GB, 28.2 in system memory	21–31 tok/s	—	Experts in system memory
Needs system memory	Qwen3-30B-A3B-Instruct-2507 Alibaba Qwen	30.5B / 3.3B	Q8_0	32.5 GB, 28.7 in system memory	21–32 tok/s	—	Experts in system memory
Needs system memory	Qwen3-30B-A3B-Thinking-2507 Alibaba Qwen	30.5B / 3.3B	Q8_0	32.5 GB, 28.7 in system memory	21–32 tok/s	—	Experts in system memory
Needs system memory	Qwen3-30B-A3B Alibaba Qwen	30.5B / 3.3B	Q8_0maker’s file	32.5 GB, 28.8 in system memory	21–32 tok/s	—	Experts in system memory
Needs system memory	Qwen3-Coder-30B-A3B-Instruct Alibaba Qwen	30.5B / 3.3B	Q8_0	32.5 GB, 28.7 in system memory	21–32 tok/s	—	Experts in system memory
Needs system memory	Hy-Memory-30B-A3B Tencent	30.1B / 3B	Q8_0maker’s file	32.0 GB, 28.6 in system memory	22–32 tok/s	—	Experts in system memory
Needs system memory	Granite Switch 4.1 30B IBM	30B	IQ4_XS	17.6 GB, 8.8 in system memory	4–7 tok/s	—	39/66 layers in system memory
Needs system memory	Qwen3.5-27B Alibaba Qwen	27.8B	IQ4_XS	14.9 GB, 6.0 in system memory	6–9 tok/s	—	28/64 layers in system memory
Needs system memory	Qwen3.6-27B Alibaba Qwen	27.8B	IQ4_XS	14.9 GB, 6.0 in system memory	6–9 tok/s	—	28/64 layers in system memory
Needs system memory	Qwen3.8-27B Alibaba Qwen	27.8B	IQ4_XS	14.9 GB, 6.0 in system memory	6–9 tok/s	—	28/64 layers in system memory
Needs system memory	Gemma 4 26B A4B IT Google	25.8B / 3.8B	BF16 (native)maker’s file	51.0 GB, 43.7 in system memory	13–19 tok/s	—	Experts in system memory
Needs system memory	Magistral Small 1.2 Mistral AI	24B	IQ4_XS	13.7 GB, 4.8 in system memory	7–11 tok/s	—	16/40 layers in system memory
Needs system memory	Devstral Small 1.0 Mistral AI	23.6B	IQ4_XS	13.5 GB, 4.7 in system memory	8–11 tok/s	—	16/40 layers in system memory
Needs system memory	Devstral Small 1.1 Mistral AI	23.6B	IQ4_XS	13.5 GB, 4.7 in system memory	8–11 tok/s	—	16/40 layers in system memory
Needs system memory	Magistral Small 1.0 Mistral AI	23.6B	IQ4_XS	13.5 GB, 4.7 in system memory	8–11 tok/s	—	16/40 layers in system memory
Needs system memory	Magistral Small 1.1 Mistral AI	23.6B	IQ4_XS	13.5 GB, 4.7 in system memory	8–11 tok/s	—	16/40 layers in system memory
Needs system memory	ERNIE-4.5-21B-A3B-Thinking Baidu	21.8B / 3B	Q8_0	23.1 GB, 14.7 in system memory	20–29 tok/s	—	19/28 layers in system memory
Needs system memory	GPT OSS 20B OpenAI	20.9B / 3.6B	MXFP4 (native)maker’s file	13.6 GB, 4.8 in system memory	42–63 tok/s	—	9/24 layers in system memory
Needs system memory	Qwen2.5-14B-Instruct-1M Alibaba Qwen	14.8B	Q4_K_M	10.4 GB, 1.4 in system memory	19–28 tok/s	—	8/48 layers in system memory
Needs system memory	Qwen2.5-14B-Instruct Alibaba Qwen	14.8B	Q4_K_Mmaker’s file	10.4 GB, 1.4 in system memory	19–28 tok/s	—	8/48 layers in system memory
Needs system memory	DeepSeek-R1-Distill-Qwen-14B DeepSeek	14.8B	Q4_K_M	10.4 GB, 1.4 in system memory	19–28 tok/s	—	8/48 layers in system memory
Needs system memory	Qwen3-14B Alibaba Qwen	14.8B	Q5_0maker’s file	11.3 GB, 2.4 in system memory	13–19 tok/s	—	10/40 layers in system memory
Needs system memory	Gemma 3 12B IT Google	12.2B	Q5_K_M	11.4 GB, 2.5 in system memory	13–20 tok/s	—	15/48 layers in system memory

Estimates from the makers’ Hugging Face repositories (as of 2026-11-06); filtered views aren’t indexed. How it’s estimated