bge-reranker-v2-m3
- Open weights
Prices
0 offerings · prices as listedNo priced offering is listed for this model yet.
Run locally
Weights: BAAI/bge-reranker-v2-m3- Parameters 0.6B
- Dense
- Native precision FP32
- Layers 24
- KV cache per token 96 KB
- Config positions 8K (config.json max_position_embeddings)
- Needs
- ≈ 3.4 GB (weights 2.1 (maker’s file) + KV cache 0.8 + overhead 0.5)
- Usable
- 21.6 GB (90% of its memory)
- Generation speed
- 164–246 tok/s
- Longest context
- 8K (the model’s limit)
- First token
- about 0.02–0.03 s for a 2K prompt
| Precision | Weights | Needs at 8K | On your hardware | Longest context |
|---|---|---|---|---|
| FP32 (native)maker’s file | 2.1 GB | ≈ 3.4 | ●Runs well 164–246 tok/s | 8K |
| Q8_0 | 0.6 GB | ≈ 1.8 | ●Runs well 358–537 tok/s | 8K |
| FP8 | 0.5 GB | ≈ 1.8 | ●Runs well 367–551 tok/s | 8K |
| Q6_K | 0.4 GB | ≈ 1.7 | ●Runs well 396–595 tok/s | 8K |
| Q5_K_M | 0.4 GB | ≈ 1.6 | ●Runs well 417–625 tok/s | 8K |
| Q5_0 | 0.4 GB | ≈ 1.6 | ●Runs well 422–632 tok/s | 8K |
| Q4_K_M | 0.3 GB | ≈ 1.6 | ●Runs well 438–657 tok/s | 8K |
| Q4_0 | 0.3 GB | ≈ 1.5 | ●Runs well 448–672 tok/s | 8K |
| IQ4_XS | 0.3 GB | ≈ 1.5 | ●Runs well 455–683 tok/s | 8K |
| Q3_K_Mquality drops | 0.3 GB | ≈ 1.5 | ●Runs well 465–698 tok/s | 8K |
| IQ3_XXSquality drops | 0.2 GB | ≈ 1.5 | ●Runs well 493–739 tok/s | 8K |
| Q2_Kquality drops | 0.2 GB | ≈ 1.4 | ●Runs well 495–742 tok/s | 8K |
| IQ2_XXSquality drops | 0.1 GB | ≈ 1.4 | ●Runs well 530–795 tok/s | 8K |
Click a row to choose a precision. “Maker’s file” sizes are the maker’s own files; others are parameters × bits per weight.
Compared with other hardware
Each at the highest precision it runs. Click one for details.
Smallest that runs it (Q4_K_M, 8K): GPU GeForce RTX 3080; unified memory Apple M5 · 16 GB.
Run commands
vllm serve BAAI/bge-reranker-v2-m3 --max-model-len 8192
Memory to fine-tune
Batch 1, 2,048-token sequences, gradient checkpointing; LoRA and QLoRA train rank-16 adapters on the attention projections.
Estimates; facts from the maker’s Hugging Face repositories, as of 2026-11-06. How it’s estimated
Cited scores
From third-party leaderboards, not measured by usNo cited leaderboard covers rerank models yet.
Changes
All changes →No changes recorded yet.
Sources
- BAAI/bge-reranker-v2-m3 model card (opens in a new tab) huggingface.co · checked Oct 30, 2026
- BAAI/bge-reranker-v2-m3 model card (opens in a new tab) huggingface.co · checked Oct 30, 2026
- BAAI/bge-reranker-v2-m3 config.json (opens in a new tab) huggingface.co · checked Nov 2, 2026
- BAAI/bge-reranker-v2-m3 files (opens in a new tab) huggingface.co · checked Oct 30, 2026
Some values on this page are still waiting to be checked against official pages: a filled dot is verified, a hollow one is not yet (with or without a source on file). How we verify data
BAAI · China · baai.ac.cn
Related models
Snapshots, the same family and other models from this provider