DeepSeek-R1-Distill-Llama-8B
- In Input modalities: Text Out Output modalities: Text
- Reasoning
- Open weights
Prices
0 offerings · prices as listedNo priced offering is listed for this model yet.
Details
- Tool use
- —
- Structured output
- —
- Licence
- llama3.1 license
- Family
- deepseek-r-distill-llama
Not stated on DeepSeek’s pages: max input, reasoning efforts, reasoning can be off, max thinking budget, built-in tools, knowledge cutoff, aliases.
Run locally
Weights: deepseek-ai/DeepSeek-R1-Distill-Llama-8B- Parameters 8B
- Dense
- Native precision BF16
- Layers 32
- KV cache per token 128 KB
- Config positions 128K (config.json max_position_embeddings)
- API context 125K (maker’s API)
- Needs
- ≈ 16.7 GB (weights 15.0 (maker’s file) + KV cache 1.0 + overhead 0.7)
- Usable
- 21.6 GB (90% of its memory)
- Generation speed
- 29–44 tok/s
- Longest context
- 47K (longer doesn’t fit)
- First token
- about 0.26–0.39 s for a 2K prompt
| Precision | Weights | Needs at 8K | On your hardware | Longest context |
|---|---|---|---|---|
| BF16 (native)maker’s file | 15.0 GB | ≈ 16.7 | ●Runs well 29–44 tok/s | 47K |
| Q8_0 | 7.9 GB | ≈ 9.4 | ●Runs well 52–79 tok/s | 105K |
| FP8 | 7.5 GB | ≈ 9.0 | ●Runs well 55–83 tok/s | 109K |
| Q6_K | 6.1 GB | ≈ 7.6 | ●Runs well 66–99 tok/s | 119K |
| Q5_K_M | 5.3 GB | ≈ 6.8 | ●Runs well 74–111 tok/s | 125K |
| Q5_0 | 5.1 GB | ≈ 6.6 | ●Runs well 76–115 tok/s | 125K |
| Q4_K_M | 4.5 GB | ≈ 6.0 | ●Runs well 85–127 tok/s | 125K |
| Q4_0 | 4.2 GB | ≈ 5.7 | ●Runs well 90–135 tok/s | 125K |
| IQ4_XS | 4.0 GB | ≈ 5.5 | ●Runs well 94–142 tok/s | 125K |
| Q3_K_Mquality drops | 3.7 GB | ≈ 5.2 | ●Runs well 101–151 tok/s | 125K |
| IQ3_XXSquality drops | 2.9 GB | ≈ 4.4 | ●Runs well 122–182 tok/s | 125K |
| Q2_Kquality drops | 2.8 GB | ≈ 4.3 | ●Runs well 123–185 tok/s | 125K |
| IQ2_XXSquality drops | 1.9 GB | ≈ 3.4 | ●Runs well 160–241 tok/s | 125K |
Click a row to choose a precision. “Maker’s file” sizes are the maker’s own files; others are parameters × bits per weight.
Compared with other hardware
Each at the highest precision it runs. Click one for details.
Smallest that runs it (Q4_K_M, 8K): GPU GeForce RTX 3080; unified memory Apple M5 · 16 GB.
Run commands
vllm serve deepseek-ai/DeepSeek-R1-Distill-Llama-8B --max-model-len 8192
Memory to fine-tune
Batch 1, 2,048-token sequences, gradient checkpointing; LoRA and QLoRA train rank-16 adapters on the attention projections.
Estimates; facts from the maker’s Hugging Face repositories, as of 2026-11-06. How it’s estimated
Cited scores
From third-party leaderboards, not measured by usThis model isn’t on any leaderboard we cite yet.
News
All news →- Sep 30, 2026Retirement Alibaba Cloud Model Studio retires DeepSeek-R1-Distill-Llama-8B DeepSeek-R1-Distill-Llama-8B is no longer available on Alibaba Cloud Model Studio from 2026-09-30.
Changes
All changes →- Available → Retired
Sources
- Alibaba Cloud Model Studio: text generation models (opens in a new tab) docs.modelstudio.console.alibabacloud.com · checked Sep 29, 2026
- models.dev: deepseek-r1-distill-llama-8b (community data, MIT) (opens in a new tab) github.com · checked Sep 30, 2026
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B model card (opens in a new tab) huggingface.co · checked Nov 2, 2026
- DeepSeek-R1 (opens in a new tab) huggingface.co · checked Oct 31, 2026
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B model card (opens in a new tab) huggingface.co · checked Nov 6, 2026
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B config.json (opens in a new tab) huggingface.co · checked Nov 2, 2026
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B files (opens in a new tab) huggingface.co · checked Oct 30, 2026
- README.md · deepseek-ai/DeepSeek-R1-Distill-Llama-8B at 81cee02dd020268dced5fa1327e8555acce9c63c (opens in a new tab) huggingface.co · checked Nov 5, 2026
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B at main (opens in a new tab) huggingface.co · checked Nov 2, 2026
DeepSeek’s official policies: data use, privacy, terms →
Some values on this page are still waiting to be checked against official pages: a filled dot is verified, a hollow one is not yet (with or without a source on file). How we verify data
DeepSeek · China · deepseek.com
Related models
Snapshots, the same family and other models from this provider