Open-weight models that run on AMD Instinct MI300X

Memory 192 GB
Bandwidth 5300 GB/s
FP16 dense 1307.4 TFLOPS
Usable 172.8 GB

Specs: Memory · AMD Instinct™ MI300X Accelerators · Bandwidth · AMD Instinct™ MI300X Accelerators · FP16 dense · AMD Instinct™ MI300X Accelerators

Search Precision Context Concurrent requests Minimum speed Architecture Type Maker Sort

Fit	Model	Parameters / active	Precision	Memory	Est. speed	Longest context	Runs as
Runs well	GLM-5.3-Flash Zhipu AI	321.3B / 18B	IQ4_XS	167.3 / 172.8 GB	267–400 tok/s	133K	One GPU
Runs well	MiMo-V2.5 Xiaomi	310.8B / 15B	Q4_0	172.1 / 172.8 GB	275–412 tok/s	13K	One GPU
Runs well	DeepSeek-V4-Flash (2026-07-31) DeepSeek	304.2B	FP8 (native)maker’s file	163.9 / 172.8 GB	138–207 tok/s	114K	One GPU
Runs well	Hy3 Tencent	298.8B / 21B	Q4_0	166.9 / 172.8 GB	183–274 tok/s	27K	One GPU
Runs well	DeepSeek-V4-Flash (2026-04-24) DeepSeek	290.9B / 13B	IQ4_XS	151.8 / 172.8 GB	347–521 tok/s	258K	One GPU
Runs well	DeepSeek-V4-Flash DeepSeek	290.9B / 13B	IQ4_XS	151.8 / 172.8 GB	347–521 tok/s	258K	One GPU
Runs well	ERNIE-5.5-Flash-Base Baidu	286.2B	IQ4_XS	149.1 / 172.8 GB	speed not estimated	510K	One GPU
Runs well	Qwen3-VL-235B-A22B-Instruct Alibaba Qwen	235.7B	Q4_K_M	141.5 / 172.8 GB	172–258 tok/s	128K	One GPU
Runs well	Qwen3-VL-235B-A22B-Thinking Alibaba Qwen	235.7B	Q4_K_M	141.5 / 172.8 GB	172–258 tok/s	128K	One GPU
Runs well	Qwen3-235B-A22B-Instruct-2507 Alibaba Qwen	235.1B / 22B	Q4_K_M	141.1 / 172.8 GB	177–266 tok/s	256K	One GPU
Runs well	Qwen3-235B-A22B-Thinking-2507 Alibaba Qwen	235.1B / 22B	Q4_K_M	141.1 / 172.8 GB	177–266 tok/s	256K	One GPU
Runs well	Qwen3-235B-A22B Alibaba Qwen	235.1B / 22B	Q4_K_Mmaker’s file	140.5 / 172.8 GB	178–267 tok/s	128K	One GPU
Runs well	MiniMax-M2.1 MiniMax	228.7B	Q4_K_M	137.8 / 172.8 GB	302–453 tok/s	152K	One GPU
Runs well	MiniMax-M2.5 MiniMax	228.7B	Q4_K_M	137.8 / 172.8 GB	302–453 tok/s	152K	One GPU
Runs well	MiniMax-M2.7 MiniMax	228.7B	Q4_K_M	137.8 / 172.8 GB	302–453 tok/s	152K	One GPU
Runs well	MiniMax-M2 MiniMax	228.7B / 10B	Q4_K_M	137.8 / 172.8 GB	325–487 tok/s	152K	One GPU
Runs well	MiniMax-M3.1-Flash MiniMax	220.5B / 23B	Q5_K_M	154.3 / 172.8 GB	153–229 tok/s	166K	One GPU
Runs well	Step 3.7 Flash StepFun	201.4B	Q5_K_M	141.5 / 172.8 GB	702–1053 tok/s	186K	One GPU
Runs well	Step 3.5 Flash StepFun	199.4B / 11B	Q5_K_M	140.1 / 172.8 GB	284–426 tok/s	194K	One GPU
Runs well	Ming-3.0-audio-flash Ant Group	128.2B / 6.5B	Q8_0	133.6 / 172.8 GB	363–544 tok/s	128K	One GPU
Runs well	Ling-3.0-flash Ant Group	127.5B / 5.1B	Q8_0maker’s file	133.0 / 172.8 GB	455–683 tok/s	250K	One GPU
Runs well	Qwen3.5-122B-A10B Alibaba Qwen	125.1B / 10B	Q8_0	130.2 / 172.8 GB	245–367 tok/s	256K	One GPU
Runs well	Ling-3.0-flash-VL Ant Group	124.8B / 5.5B	Q8_0	130.1 / 172.8 GB	425–637 tok/s	128K	One GPU
Runs well	gpt-oss-120b OpenAI	116.8B / 5.1B	MXFP4 (native)maker’s file	64.1 / 172.8 GB	840–1260 tok/s	128K	One GPU
Runs well	GLM-4.5-Air Zhipu AI	110.5B	Q8_0	116.2 / 172.8 GB	135–202 tok/s	125K	One GPU
Runs well	GLM-4.5V Zhipu AI	107.7B / 12B	Q8_0	113.3 / 172.8 GB	185–278 tok/s	63K	One GPU
Runs well	GLM-4.6V Zhipu AI	107.7B	Q8_0	113.3 / 172.8 GB	159–238 tok/s	125K	One GPU
Runs well	Ling-2.6-flash Ant Group	107.5B / 7.4B	Q8_0	112.0 / 172.8 GB	325–487 tok/s	128K	One GPU
Runs well	Qwen3-Next-80B-A3B-Instruct Alibaba Qwen	81.3B / 3B	Q8_0maker’s file	83.1 / 172.8 GB	796–1194 tok/s	256K	One GPU
Runs well	Qwen3-Next-80B-A3B-Thinking Alibaba Qwen	81.3B / 3B	Q8_0maker’s file	83.1 / 172.8 GB	796–1194 tok/s	256K	One GPU
Runs well	Qwen3-Coder-Next Alibaba Qwen	79.7B / 3B	Q8_0maker’s file	83.1 / 172.8 GB	781–1171 tok/s	256K	One GPU
Runs well	Qwen2.5-VL-72B-Instruct Alibaba Qwen	73.4B	BF16 (native)maker’s file	146.1 / 172.8 GB	18–27 tok/s	94K	One GPU
Runs well	Qwen2-VL-72B-Instruct Alibaba Qwen	73.4B	BF16 (native)maker’s file	146.1 / 172.8 GB	18–27 tok/s	32K	One GPU
Runs well	Qwen2.5-72B-Instruct Alibaba Qwen	72.7B	BF16 (native)maker’s file	144.7 / 172.8 GB	18–27 tok/s	32K	One GPU
Runs well	Qianfan-VL-70B Baidu	71.8B	BF16 (native)maker’s file	143.0 / 172.8 GB	18–27 tok/s	32K	One GPU
Runs well	DeepSeek-R1-Distill-Llama-70B DeepSeek	70.6B	BF16 (native)maker’s file	140.5 / 172.8 GB	18–28 tok/s	111K	One GPU
Runs well	Qwen3.5-35B-A3B Alibaba Qwen	36B / 3B	BF16 (native)maker’s file	70.5 / 172.8 GB	430–644 tok/s	256K	One GPU
Runs well	Qwen3.6-35B-A3B Alibaba Qwen	36B / 3B	BF16 (native)maker’s file	70.5 / 172.8 GB	430–644 tok/s	256K	One GPU
Runs well	Qwen2.5-VL-32B-Instruct Alibaba Qwen	33.5B	BF16 (native)maker’s file	68.8 / 172.8 GB	38–56 tok/s	125K	One GPU
Runs well	Qwen3-VL-32B-Instruct Alibaba Qwen	33.4B	BF16 (native)maker’s file	67.2 / 172.8 GB	38–58 tok/s	128K	One GPU
Runs well	Qwen3-VL-32B-Thinking Alibaba Qwen	33.4B	BF16 (native)maker’s file	67.2 / 172.8 GB	38–58 tok/s	128K	One GPU
Runs well	Qwen2.5-32B-Instruct Alibaba Qwen	32.8B	BF16 (native)maker’s file	66.1 / 172.8 GB	39–59 tok/s	32K	One GPU
Runs well	Qwen2.5-Coder-32B-Instruct Alibaba Qwen	32.8B	BF16 (native)maker’s file	66.1 / 172.8 GB	39–59 tok/s	32K	One GPU
Runs well	QwQ-32B Alibaba Qwen	32.8B	BF16 (native)maker’s file	66.1 / 172.8 GB	39–59 tok/s	40K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-32B DeepSeek	32.8B	BF16 (native)maker’s file	66.1 / 172.8 GB	39–59 tok/s	32K	One GPU
Runs well	Qwen3-32B Alibaba Qwen	32.8B	BF16 (native)maker’s file	66.1 / 172.8 GB	39–59 tok/s	128K	One GPU
Runs well	Qwen3-Omni-30B-A3B-Captioner Alibaba Qwen	31.7B	BF16 (native)maker’s file	62.8 / 172.8 GB	268–402 tok/s	64K	One GPU
Runs well	Gemma 4 31B IT Google	31.3B	BF16 (native)maker’s file	63.2 / 172.8 GB	41–61 tok/s	250K	One GPU
Runs well	GLM-4.7-Flash Zhipu AI	31.2B	BF16 (native)maker’s file	61.5 / 172.8 GB	246–368 tok/s	195K	One GPU
Runs well	Qwen3-VL-30B-A3B-Instruct Alibaba Qwen	31.1B	BF16 (native)maker’s file	61.5 / 172.8 GB	309–463 tok/s	128K	One GPU
Runs well	Qwen3-VL-30B-A3B-Thinking Alibaba Qwen	31.1B	BF16 (native)maker’s file	61.5 / 172.8 GB	309–463 tok/s	128K	One GPU
Runs well	Qwen3-30B-A3B-Instruct-2507 Alibaba Qwen	30.5B / 3.3B	BF16 (native)maker’s file	60.5 / 172.8 GB	358–537 tok/s	256K	One GPU
Runs well	Qwen3-30B-A3B-Thinking-2507 Alibaba Qwen	30.5B / 3.3B	BF16 (native)maker’s file	60.5 / 172.8 GB	358–537 tok/s	256K	One GPU
Runs well	Qwen3-30B-A3B Alibaba Qwen	30.5B / 3.3B	BF16 (native)maker’s file	60.5 / 172.8 GB	358–537 tok/s	128K	One GPU
Runs well	Qwen3-Coder-30B-A3B-Instruct Alibaba Qwen	30.5B / 3.3B	BF16 (native)maker’s file	60.5 / 172.8 GB	358–537 tok/s	256K	One GPU
Runs well	Hy-Memory-30B-A3B Tencent	30.1B / 3B	BF16 (native)maker’s file	59.6 / 172.8 GB	389–584 tok/s	63K	One GPU
Runs well	Granite Switch 4.1 30B IBM	30B	BF16 (native)maker’s file	60.7 / 172.8 GB	43–64 tok/s	128K	One GPU
Runs well	Qwen3.5-27B Alibaba Qwen	27.8B	BF16 (native)maker’s file	54.8 / 172.8 GB	47–71 tok/s	256K	One GPU
Runs well	Qwen3.6-27B Alibaba Qwen	27.8B	BF16 (native)maker’s file	54.8 / 172.8 GB	47–71 tok/s	256K	One GPU
Runs well	Qwen3.8-27B Alibaba Qwen	27.8B	BF16 (native)maker’s file	54.8 / 172.8 GB	47–71 tok/s	256K	One GPU
Runs well	Gemma 4 26B A4B IT Google	25.8B / 3.8B	BF16 (native)maker’s file	51.0 / 172.8 GB	325–488 tok/s	250K	One GPU
Runs well	Magistral Small 1.2 Mistral AI	24B	BF16 (native)maker’s file	95.2 / 172.8 GB	27–41 tok/s	125K	One GPU
Runs well	Devstral Small 1.0 Mistral AI	23.6B	BF16 (native)maker’s file	93.5 / 172.8 GB	28–42 tok/s	125K	One GPU
Runs well	Devstral Small 1.1 Mistral AI	23.6B	BF16 (native)maker’s file	93.5 / 172.8 GB	28–42 tok/s	125K	One GPU
Runs well	Magistral Small 1.0 Mistral AI	23.6B	BF16 (native)maker’s file	93.5 / 172.8 GB	28–42 tok/s	39K	One GPU
Runs well	Magistral Small 1.1 Mistral AI	23.6B	BF16 (native)maker’s file	93.5 / 172.8 GB	28–42 tok/s	39K	One GPU
Runs well	ERNIE-4.5-21B-A3B-Thinking Baidu	21.8B / 3B	BF16 (native)maker’s file	43.1 / 172.8 GB	410–614 tok/s	128K	One GPU
Runs well	GPT OSS 20B OpenAI	20.9B / 3.6B	MXFP4 (native)maker’s file	13.6 / 172.8 GB	1030–1545 tok/s	128K	One GPU
Runs well	Qwen2.5-14B-Instruct-1M Alibaba Qwen	14.8B	BF16 (native)maker’s file	30.4 / 172.8 GB	85–128 tok/s	767K	One GPU
Runs well	Qwen2.5-14B-Instruct Alibaba Qwen	14.8B	BF16 (native)maker’s file	30.4 / 172.8 GB	85–128 tok/s	32K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-14B DeepSeek	14.8B	BF16 (native)maker’s file	30.4 / 172.8 GB	85–128 tok/s	32K	One GPU
Runs well	Qwen3-14B Alibaba Qwen	14.8B	BF16 (native)maker’s file	30.1 / 172.8 GB	86–129 tok/s	128K	One GPU
Runs well	Gemma 3 12B IT Google	12.2B	BF16 (native)maker’s file	26.6 / 172.8 GB	97–145 tok/s	128K	One GPU
Runs well	Qwen2.5-Omni-7B Alibaba Qwen	10.7B	BF16 (native)maker’s file	22.3 / 172.8 GB	116–174 tok/s	32K	One GPU
Runs well	GLM-4.6V-Flash Zhipu AI	10.3B	BF16 (native)maker’s file	20.4 / 172.8 GB	127–190 tok/s	125K	One GPU
Runs well	Qwen3.5-9B Alibaba Qwen	9.7B	BF16 (native)maker’s file	19.1 / 172.8 GB	135–203 tok/s	256K	One GPU
Runs well	Qianfan-VL-8B Baidu	8.8B	BF16 (native)maker’s file	18.2 / 172.8 GB	142–213 tok/s	32K	One GPU
Runs well	Granite Switch 4.1 8B IBM	8.8B	BF16 (native)maker’s file	18.5 / 172.8 GB	140–210 tok/s	128K	One GPU
Runs well	Qwen3-VL-8B-Instruct Alibaba Qwen	8.8B	BF16 (native)maker’s file	18.3 / 172.8 GB	141–212 tok/s	128K	One GPU
Runs well	Qwen3-VL-8B-Thinking Alibaba Qwen	8.8B	BF16 (native)maker’s file	18.3 / 172.8 GB	141–212 tok/s	128K	One GPU
Runs well	pplx-embed-v2-context-9b Perplexity	8.4B	FP32 (native)maker’s file	33.1 / 172.8 GB	78–117 tok/s	256K	One GPU
Runs well	Qwen2.5-VL-7B-Instruct Alibaba Qwen	8.3B	BF16 (native)maker’s file	16.7 / 172.8 GB	155–233 tok/s	125K	One GPU
Runs well	Qwen3-8B Alibaba Qwen	8.2B	BF16 (native)maker’s file	17.1 / 172.8 GB	151–226 tok/s	128K	One GPU
Runs well	DeepSeek-R1-Distill-Llama-8B DeepSeek	8B	BF16 (native)maker’s file	16.7 / 172.8 GB	155–232 tok/s	125K	One GPU
Runs well	Gemma 4 E4B Google	8B	BF16 (native)maker’s file	15.8 / 172.8 GB	164–246 tok/s	125K	One GPU
Runs well	Ling-3.0-tiny Ant Group	7.9B / 1.3B	BF16 (native)maker’s file	15.6 / 172.8 GB	937–1406 tok/s	128K	One GPU
Runs well	Qwen2.5-7B-Instruct-1M Alibaba Qwen	7.6B	BF16 (native)maker’s file	15.3 / 172.8 GB	169–253 tok/s	977K	One GPU
Runs well	Qwen2.5-7B-Instruct Alibaba Qwen	7.6B	BF16 (native)maker’s file	15.3 / 172.8 GB	169–253 tok/s	32K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-7B DeepSeek	7.6B	BF16 (native)maker’s file	15.3 / 172.8 GB	169–253 tok/s	32K	One GPU
Runs well	Mistral-7B-Instruct-v0.1 Mistral AI	7.2B	BF16 (native)maker’s file	15.2 / 172.8 GB	170–256 tok/s	32K	One GPU
Runs well	Gemma 4 E2B Google	5.1B	BF16 (native)maker’s file	10.1 / 172.8 GB	257–385 tok/s	125K	One GPU
Runs well	WeVisOne-VL-4B Tencent	4.8B	BF16 (native)maker’s file	10.6 / 172.8 GB	244–366 tok/s	256K	One GPU
Runs well	Qianfan-OCR Baidu	4.7B	BF16 (native)maker’s file	10.5 / 172.8 GB	248–372 tok/s	31K	One GPU
Runs well	Qwen3.5-4B Alibaba Qwen	4.7B	BF16 (native)maker’s file	9.4 / 172.8 GB	276–415 tok/s	256K	One GPU
Runs well	Qwen3-VL-4B-Instruct Alibaba Qwen	4.4B	BF16 (native)maker’s file	9.9 / 172.8 GB	263–394 tok/s	250K	One GPU
Runs well	Qwen3-4B-Instruct-2507 Alibaba Qwen	4B	BF16 (native)maker’s file	9.1 / 172.8 GB	286–430 tok/s	256K	One GPU
Runs well	Qwen3-4B Alibaba Qwen	4B	BF16 (native)maker’s file	9.1 / 172.8 GB	286–430 tok/s	128K	One GPU
Runs well	DeepSeek-OCR DeepSeek	3.3B	BF16 (native)maker’s file	7.2 / 172.8 GB	951–1427 tok/s	8K	One GPU
Runs well	Granite-4.0-H-Micro IBM	3.2B	BF16 (native)maker’s file	6.5 / 172.8 GB	411–616 tok/s	128K	One GPU
Runs well	Qwen3-1.7B Alibaba Qwen	2B	BF16 (native)maker’s file	5.2 / 172.8 GB	530–795 tok/s	40K	One GPU
Runs well	DeepSeek-R1-Distill-Qwen-1.5B DeepSeek	1.8B	BF16 (native)maker’s file	4.0 / 172.8 GB	699–1049 tok/s	125K	One GPU
Runs well	GLM-OCR Zhipu AI	1.3B	BF16 (native)maker’s file	3.5 / 172.8 GB	831–1247 tok/s	128K	One GPU
Runs well	Qwen3-0.6B Alibaba Qwen	0.8B	BF16 (native)maker’s file	2.8 / 172.8 GB	1085–1627 tok/s	40K	One GPU
Runs well	EmbeddingGemma 2 Google	0.7B	BF16 (native)maker’s file	2.0 / 172.8 GB	1680–2521 tok/s	256K	One GPU
Runs well	Qwen3-Embedding-0.6B Alibaba Qwen	0.6B	BF16 (native)maker’s file	2.5 / 172.8 GB	1243–1865 tok/s	32K	One GPU
Runs well	bge-reranker-v2-m3 BAAI	0.6B	FP32 (native)maker’s file	3.4 / 172.8 GB	861–1292 tok/s	8K	One GPU
Runs well	bge-m3-unsupervised BAAI	0.6B	FP32 (native)maker’s file	3.4 / 172.8 GB	861–1292 tok/s	8K	One GPU
Runs well	bge-large-en-v1.5 BAAI	0.3B	FP32 (native)maker’s file	1.8 / 172.8 GB	1905–2858 tok/s	512	One GPU
Runs well	bge-reranker-base BAAI	0.3B	FP32 (native)maker’s file	1.6 / 172.8 GB	2343–3514 tok/s	514	One GPU
Runs well	bge-base-en-v1.5 BAAI	0.1B	FP32 (native)maker’s file	0.9 / 172.8 GB	5801–8701 tok/s	512	One GPU
Runs well	bge-small-en-v1.5 BAAI	0B	FP32 (native)maker’s file	0.6 / 172.8 GB	18544–27816 tok/s	512	One GPU
Needs system memory	MiniMax-M3 MiniMax	427B / 23B	IQ4_XS	222.8 GB, 52.8 in system memory	14–21 tok/s	—	15/60 layers in system memory
Needs system memory	Qwen3.5-397B-A17B Alibaba Qwen	403.4B / 17B	Q4_0	222.1 GB, 52.8 in system memory	18–27 tok/s	—	15/60 layers in system memory
Needs system memory	GLM-4.5 Zhipu AI	358.3B	IQ4_XS	189.0 GB, 17.3 in system memory	19–29 tok/s	—	9/92 layers in system memory
Needs system memory	GLM-4.7 Zhipu AI	358.3B	IQ4_XS	189.0 GB, 17.3 in system memory	19–29 tok/s	—	9/92 layers in system memory
Needs system memory	GLM-4.6 Zhipu AI	356.8B	Q4_0	199.1 GB, 26.4 in system memory	14–20 tok/s	—	13/92 layers in system memory

Estimates from the makers’ Hugging Face repositories (as of 2026-11-06); filtered views aren’t indexed. How it’s estimated