AI Index

Magistral Small 1.2

Language Mistral AI Partly verified Retired Shut down Jul 31, 2026 Released Sep 17, 2025
  • In TextImage Out Text
  • Reasoning
  • Tool use
  • Open weights

Prices

0 offerings · prices as listed

No priced offering is listed for this model yet.

Details

Licence
Apache 2.0
Family
magistral-small

Not stated on Mistral AI’s pages: max input, reasoning efforts, reasoning can be off, max thinking budget, structured output, built-in tools, knowledge cutoff, aliases.

Run locally

Weights: mistralai/Magistral-Small-2509
  • Parameters 24B
  • Dense
  • Native precision BF16
  • Layers 40
  • KV cache per token 160 KB
  • Config positions 128K (config.json max_position_embeddings)
  • API context 125K (maker’s API)
My hardware GeForce RTX 4090 24 GB · 1008 GB/s · 64 GB DDR5 dual channel · 89.6 GB/s
On your hardware · GeForce RTX 4090
Runs well Q5_K_M (automatic) 8K context 28–42 tok/sOne GPU
WeightsKV cacheOverheadUsable
Needs
≈ 17.6 GB (weights 15.6 (maker’s file) + KV cache 1.3 + overhead 0.8)
Usable
21.6 GB (90% of its memory)
Generation speed
28–42 tok/s
Longest context
33K (longer doesn’t fit)
First token
about 0.78–1.16 s for a 2K prompt
PrecisionWeightsNeeds at 8KOn your hardwareLongest context
BF16 (native)maker’s file89.4 GB≈ 95.2Doesn’t fit—
Q8_0maker’s file23.3 GB≈ 25.8Needs system memory 7–10 tok/s—
FP822.4 GB≈ 24.7Needs system memory 8–12 tok/s—
Q6_K18.4 GB≈ 20.5Runs well 24–36 tok/s15K
Q5_K_Mmaker’s file15.6 GB≈ 17.6Runs well 28–42 tok/s33K
Q5_015.4 GB≈ 17.4Runs well 28–42 tok/s35K
Q4_K_Mmaker’s file13.3 GB≈ 15.3Runs well 32–48 tok/s48K
Q4_012.6 GB≈ 14.5Runs well 34–51 tok/s53K
IQ4_XS11.9 GB≈ 13.7Runs well 36–54 tok/s58K
Q3_K_Mquality drops10.9 GB≈ 12.7Runs well 39–58 tok/s64K
IQ3_XXSquality drops8.6 GB≈ 10.3Runs well 48–72 tok/s80K
Q2_Kquality drops8.4 GB≈ 10.1Runs well 49–73 tok/s81K
IQ2_XXSquality drops5.8 GB≈ 7.5Runs well 67–100 tok/s98K

Click a row to choose a precision. “Maker’s file” sizes are the maker’s own files; others are parameters × bits per weight.

Compared with other hardware

Each at the highest precision it runs. Click one for details.

Smallest that runs it (Q4_K_M, 8K): GPU GeForce RTX 4090; unified memory Apple M5 Pro (16-core GPU) · 24 GB.

Run commands

llama.cpp
llama-server -hf mistralai/Magistral-Small-2509-GGUF:Q5_K_M -c 8192
Ollama
ollama run hf.co/mistralai/Magistral-Small-2509-GGUF:Q5_K_M
Memory to fine-tune
Full≈ 360.4 GBDoesn’t fit your hardware
LoRA≈ 47.6 GBDoesn’t fit your hardware
QLoRA≈ 15.5 GBFits your hardware

Batch 1, 2,048-token sequences, gradient checkpointing; LoRA and QLoRA train rank-16 adapters on the attention projections.

Estimates; facts from the maker’s Hugging Face repositories, as of 2026-11-06. How it’s estimated

Cited scores

From third-party leaderboards, not measured by us
  • Epoch AI · GPQA Diamond 47.6%

    #133 of 168 listed models · ±2.6

  • Epoch AI · OTIS Mock AIME 2024–2025 28.1%

    #123 of 159 listed models · ±5.56

  • Epoch AI, 'Capabilities & benchmarking'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/benchmarks'. Licensed CC BY 4.0. · License · as of Nov 4, 2026
  • changes we made

Changes

All changes →

No changes recorded yet.

Sources

Mistral AI’s official policies: data use, privacy, terms →

Some values on this page are still waiting to be checked against official pages: a filled dot is verified, a hollow one is not yet (with or without a source on file). How we verify data

Mistral AI · France · mistral.ai

Snapshots, the same family and other models from this provider