Model field guide
SmolLM2-360M-Instruct
42 of 44 hardware configurations have a supported memory fit at one or more recorded quantizations. Runtime compatibility still needs verification.
Total parameters0.36B
Active parameters0.36B
Recorded context limit8,192
Architecturedense
Verified 2026-10-22 · license metadata: apache-2.0 · commercial use: yes (review the terms).
Actual files, not a size shortcut.
| Quantization | Total file size | Source |
|---|---|---|
| Q8_0 | 0.360 GiB386,404,992 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q4_K_M | 0.252 GiB270,590,560 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q5_K_M | 0.270 GiB289,944,160 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| FP16 | 0.676 GiB725,553,760 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
The hardware picture.
Q8_0: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 1.52 GiB | fits | 158.4–316.8 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 1.52 GiB | fits | 176.0–352.0 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 1.52 GiB | fits | 176.0–352.0 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 1.52 GiB | fits | 317.8–635.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 1.52 GiB | fits | 317.8–635.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 1.52 GiB | fits | 317.8–635.6 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 1.52 GiB | fits | 476.2–952.4 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 1.52 GiB | fits | 635.6–1271.2 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.52 GiB | fits | 635.6–1271.2 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 1.52 GiB | fits | 1242.2–2484.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.52 GiB | fits | 1242.2–2484.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 1.52 GiB | fits | 1242.2–2484.4 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 1.52 GiB | fits | 158.4–316.8 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 1.52 GiB | fits | 317.8–635.6 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.52 GiB | fits | 635.6–1271.2 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 1.52 GiB | fits | 124.2–248.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 1.52 GiB | fits | 124.2–248.4 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 1.52 GiB | fits | 124.2–248.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 1.52 GiB | fits | 282.6–565.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 1.52 GiB | fits | 282.6–565.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 1.52 GiB | fits | 282.6–565.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 1.52 GiB | fits | 424.4–848.9 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 1.52 GiB | fits | 565.2–1130.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 1.52 GiB | fits | 565.2–1130.4 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 1.52 GiB | fits | 847.8–1695.6 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.52 GiB | fits | 847.8–1695.6 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 1.52 GiB | fits | 372.7–745.3 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 1.52 GiB | fits | 968.9–1937.9 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 1.52 GiB | fits | 298.1–596.3 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 1.52 GiB | fits | 521.7–1043.5 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 1.52 GiB | fits | 742.0–1484.0 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 1.52 GiB | fits | 1043.5–2086.9 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 1.52 GiB | fits | 463.8–927.5 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 1.52 GiB | fits | 695.6–1391.3 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 1.52 GiB | fits | 927.5–1855.0 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 1.52 GiB | fits | 993.8–1987.6 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 1.52 GiB | fits | 1855.0–3710.1 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 1.52 GiB | fits | 1855.0–3710.1 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 1.52 GiB | fits | 993.8–1987.6 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 1.52 GiB | fits | 472.0–944.1 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 1.52 GiB | fits | 265.0–530.0 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 1.52 GiB | fits | 282.6–565.2 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 1.52 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 1.52 GiB | Needs verification | Estimated speed unavailable |
Q4_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 1.41 GiB | fits | 226.2–452.3 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 1.41 GiB | fits | 251.3–502.6 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 1.41 GiB | fits | 251.3–502.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 1.41 GiB | fits | 453.8–907.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 1.41 GiB | fits | 453.8–907.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 1.41 GiB | fits | 453.8–907.6 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 1.41 GiB | fits | 680.0–1360.0 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 1.41 GiB | fits | 907.6–1815.3 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.41 GiB | fits | 907.6–1815.3 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 1.41 GiB | fits | 1773.9–3547.8 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.41 GiB | fits | 1773.9–3547.8 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 1.41 GiB | fits | 1773.9–3547.8 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 1.41 GiB | fits | 226.2–452.3 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 1.41 GiB | fits | 453.8–907.6 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.41 GiB | fits | 907.6–1815.3 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 1.41 GiB | fits | 177.4–354.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 1.41 GiB | fits | 177.4–354.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 1.41 GiB | fits | 177.4–354.8 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 1.41 GiB | fits | 403.6–807.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 1.41 GiB | fits | 403.6–807.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 1.41 GiB | fits | 403.6–807.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 1.41 GiB | fits | 606.1–1212.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 1.41 GiB | fits | 807.1–1614.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 1.41 GiB | fits | 807.1–1614.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 1.41 GiB | fits | 1210.7–2421.4 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.41 GiB | fits | 1210.7–2421.4 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 1.41 GiB | fits | 532.2–1064.3 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 1.41 GiB | fits | 1383.6–2767.3 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 1.41 GiB | fits | 425.7–851.5 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 1.41 GiB | fits | 745.0–1490.1 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 1.41 GiB | fits | 1059.6–2119.2 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 1.41 GiB | fits | 1490.1–2980.1 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 1.41 GiB | fits | 662.3–1324.5 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 1.41 GiB | fits | 993.4–1986.8 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 1.41 GiB | fits | 1324.5–2649.0 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 1.41 GiB | fits | 1419.1–2838.2 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 1.41 GiB | fits | 2649.0–5298.0 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 1.41 GiB | fits | 2649.0–5298.0 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 1.41 GiB | fits | 1419.1–2838.2 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 1.41 GiB | fits | 674.1–1348.2 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 1.41 GiB | fits | 378.4–756.9 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 1.41 GiB | fits | 403.6–807.1 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 1.41 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 1.41 GiB | Needs verification | Estimated speed unavailable |
Q5_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 1.43 GiB | fits | 211.1–422.2 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 1.43 GiB | fits | 234.5–469.1 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 1.43 GiB | fits | 234.5–469.1 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 1.43 GiB | fits | 423.5–847.1 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 1.43 GiB | fits | 423.5–847.1 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 1.43 GiB | fits | 423.5–847.1 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 1.43 GiB | fits | 634.6–1269.2 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 1.43 GiB | fits | 847.1–1694.1 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.43 GiB | fits | 847.1–1694.1 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 1.43 GiB | fits | 1655.5–3311.0 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.43 GiB | fits | 1655.5–3311.0 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 1.43 GiB | fits | 1655.5–3311.0 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 1.43 GiB | fits | 211.1–422.2 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 1.43 GiB | fits | 423.5–847.1 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.43 GiB | fits | 847.1–1694.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 1.43 GiB | fits | 165.5–331.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 1.43 GiB | fits | 165.5–331.1 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 1.43 GiB | fits | 165.5–331.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 1.43 GiB | fits | 376.6–753.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 1.43 GiB | fits | 376.6–753.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 1.43 GiB | fits | 376.6–753.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 1.43 GiB | fits | 565.6–1131.3 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 1.43 GiB | fits | 753.2–1506.5 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 1.43 GiB | fits | 753.2–1506.5 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 1.43 GiB | fits | 1129.9–2259.7 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.43 GiB | fits | 1129.9–2259.7 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 1.43 GiB | fits | 496.6–993.3 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 1.43 GiB | fits | 1291.3–2582.6 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 1.43 GiB | fits | 397.3–794.6 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 1.43 GiB | fits | 695.3–1390.6 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 1.43 GiB | fits | 988.9–1977.8 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 1.43 GiB | fits | 1390.6–2781.2 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 1.43 GiB | fits | 618.1–1236.1 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 1.43 GiB | fits | 927.1–1854.2 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 1.43 GiB | fits | 1236.1–2472.2 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 1.43 GiB | fits | 1324.4–2648.8 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 1.43 GiB | fits | 2472.2–4944.4 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 1.43 GiB | fits | 2472.2–4944.4 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 1.43 GiB | fits | 1324.4–2648.8 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 1.43 GiB | fits | 629.1–1258.2 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 1.43 GiB | fits | 353.2–706.3 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 1.43 GiB | fits | 376.6–753.2 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 1.43 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 1.43 GiB | Needs verification | Estimated speed unavailable |
FP16: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 1.83 GiB | fits | 84.3–168.7 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 1.83 GiB | fits | 93.7–187.4 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 1.83 GiB | fits | 93.7–187.4 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 1.83 GiB | fits | 169.3–338.5 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 1.83 GiB | fits | 169.3–338.5 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 1.83 GiB | fits | 169.3–338.5 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 1.83 GiB | fits | 253.6–507.2 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 1.83 GiB | fits | 338.5–677.0 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.83 GiB | fits | 338.5–677.0 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 1.83 GiB | fits | 661.6–1323.1 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.83 GiB | fits | 661.6–1323.1 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 1.83 GiB | fits | 661.6–1323.1 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 1.83 GiB | fits | 84.3–168.7 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 1.83 GiB | fits | 169.3–338.5 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 1.83 GiB | fits | 338.5–677.0 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 1.83 GiB | fits | 66.2–132.3 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 1.83 GiB | fits | 66.2–132.3 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 1.83 GiB | fits | 66.2–132.3 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 1.83 GiB | fits | 150.5–301.0 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 1.83 GiB | fits | 150.5–301.0 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 1.83 GiB | fits | 150.5–301.0 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 1.83 GiB | fits | 226.0–452.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 1.83 GiB | fits | 301.0–602.0 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 1.83 GiB | fits | 301.0–602.0 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 1.83 GiB | fits | 451.5–903.0 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 1.83 GiB | fits | 451.5–903.0 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 1.83 GiB | fits | 198.5–396.9 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 1.83 GiB | fits | 516.0–1032.0 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 1.83 GiB | fits | 158.8–317.6 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 1.83 GiB | fits | 277.9–555.7 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 1.83 GiB | fits | 395.2–790.3 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 1.83 GiB | fits | 555.7–1111.4 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 1.83 GiB | fits | 247.0–494.0 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 1.83 GiB | fits | 370.5–741.0 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 1.83 GiB | fits | 494.0–987.9 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 1.83 GiB | fits | 529.3–1058.5 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 1.83 GiB | fits | 987.9–1975.9 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 1.83 GiB | fits | 987.9–1975.9 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 1.83 GiB | fits | 529.3–1058.5 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 1.83 GiB | fits | 251.4–502.8 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 1.83 GiB | fits | 141.1–282.3 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 1.83 GiB | fits | 150.5–301.0 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 1.83 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 1.83 GiB | Needs verification | Estimated speed unavailable |
A “fits” result checks estimated memory only. “Needs verification” preserves missing cache, vision, or placement information. All speed figures are estimated from the published bandwidth formula.
Field-level sources and verification dates
- id: Official source · verified 2026-10-22. Lowercase URL slug derived from the upstream repository ID; baseModel retains the exact Hugging Face identifier.
- name: Official source · verified 2026-10-22
- baseModel: Official source · verified 2026-10-22
- downloads: Official source · verified 2026-10-22
- tasks: Official source · verified 2026-10-22. Chat denotes a text-generating model, including base models that may require a chat template or instruction tuning. Code label is inferred from the repository model-family name. Vision uses pipeline metadata or vision_config.
- license: Official source · verified 2026-10-22
- commercialUse: Official source · verified 2026-10-22. Official upstream organization metadata identifies a permissive standard license. Commercial use is permitted subject to the license terms, notices, and applicable restrictions; this is a metadata-based classification, not an individualized legal approval.
- license_url: Official source · verified 2026-10-22. No recognized standalone license file; linked pinned model card.
- architecture: Official source · verified 2026-10-22. Classification from model_type and expert/state-space configuration fields (llama); hybrid classification takes precedence over expert count.
- parametersB: Official source · verified 2026-10-22. Exact safetensors parameter total reported by the Hugging Face API, divided by 1e9.
- activeParametersB: Official source · verified 2026-10-22. Dense model: all parameters are included. This is not a runtime FLOP measurement.
- contextLength: Official source · verified 2026-10-22. Configured maximum token positions; not a guarantee of runtime support or quality at that length. No speculative RoPE scaling applied.
- kvCache: Official source · verified 2026-10-22. Standard attention KV shape from num_hidden_layers, num_key_value_heads, and head_dim or hidden_size / num_attention_heads; FP16 cache bytes are a separate estimator choice.
- licenseStandard: Official source · verified 2026-10-22. Official standard license text supporting the permissive-license classification; model applicability is identified separately by upstream metadata.