Model field guide
DeepSeek-R1-Distill-Llama-8B
42 of 44 hardware configurations have a supported memory fit at one or more recorded quantizations. Runtime compatibility still needs verification.
Total parameters8.03B
Active parameters8.03B
Recorded context limit131,072
Architecturedense
Verified 2026-10-22 · license metadata: mit · commercial use: conditional (review the terms).
- Conditional commercial use: the upstream model card, section 7, identifies MIT terms and derivation from Llama 3.1 8B under the Llama 3.1 license. Review and comply with the inherited Llama terms in addition to MIT; the MIT metadata tag alone is incomplete.
Actual files, not a size shortcut.
| Quantization | Total file size | Source |
|---|---|---|
| Q4_K_M | 4.583 GiB4,920,737,216 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q5_K_M | 5.339 GiB5,732,990,400 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| Q8_0 | 7.954 GiB8,540,773,088 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
| FP16 | 14.965 GiB16,068,893,408 bytes; 1 file(s)Sum of exact file byte sizes in the repository listing at the recorded commit; every declared shard is present. Auxiliary vision projector and speculative-decoding files are excluded. Labels follow publisher filenames; mixed-precision expert tensors may remain in a differently labeled file, so the estimator uses measured bytes rather than assumed bits per weight. | Pinned file recordVerified 2026-10-22 |
The hardware picture.
Q4_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 6.08 GiB | fits | 12.4–24.9 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 6.08 GiB | fits | 13.8–27.6 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 6.08 GiB | fits | 13.8–27.6 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 6.08 GiB | fits | 25.0–49.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 6.08 GiB | fits | 25.0–49.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 6.08 GiB | fits | 25.0–49.9 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 6.08 GiB | fits | 37.4–74.8 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 6.08 GiB | fits | 49.9–99.8 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 6.08 GiB | fits | 49.9–99.8 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 6.08 GiB | fits | 97.5–195.1 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 6.08 GiB | fits | 97.5–195.1 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 6.08 GiB | fits | 97.5–195.1 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 6.08 GiB | fits | 12.4–24.9 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 6.08 GiB | fits | 25.0–49.9 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 6.08 GiB | fits | 49.9–99.8 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 6.08 GiB | fits | 9.8–19.5 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 6.08 GiB | fits | 9.8–19.5 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 6.08 GiB | fits | 9.8–19.5 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 6.08 GiB | fits | 22.2–44.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 6.08 GiB | fits | 22.2–44.4 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 6.08 GiB | fits | 22.2–44.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 6.08 GiB | fits | 33.3–66.7 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 6.08 GiB | fits | 44.4–88.8 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 6.08 GiB | fits | 44.4–88.8 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 6.08 GiB | fits | 66.6–133.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 6.08 GiB | fits | 66.6–133.2 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 6.08 GiB | fits | 29.3–58.5 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 6.08 GiB | fits | 76.1–152.2 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 6.08 GiB | fits | 23.4–46.8 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 6.08 GiB | fits | 41.0–81.9 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 6.08 GiB | fits | 58.3–116.5 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 6.08 GiB | fits | 81.9–163.9 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 6.08 GiB | fits | 36.4–72.8 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 6.08 GiB | fits | 54.6–109.3 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 6.08 GiB | fits | 72.8–145.7 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 6.08 GiB | fits | 78.0–156.1 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 6.08 GiB | fits | 145.7–291.3 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 6.08 GiB | fits | 145.7–291.3 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 6.08 GiB | fits | 78.0–156.1 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 6.08 GiB | fits | 37.1–74.1 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 6.08 GiB | fits | 20.8–41.6 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 6.08 GiB | fits | 22.2–44.4 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 6.08 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 6.08 GiB | Needs verification | Estimated speed unavailable |
Q5_K_M: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 6.84 GiB | fits | 10.7–21.4 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 6.84 GiB | fits | 11.9–23.7 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 6.84 GiB | fits | 11.9–23.7 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 6.84 GiB | fits | 21.4–42.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 6.84 GiB | fits | 21.4–42.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 6.84 GiB | fits | 21.4–42.8 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 6.84 GiB | fits | 32.1–64.2 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 6.84 GiB | fits | 42.8–85.7 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 6.84 GiB | fits | 42.8–85.7 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 6.84 GiB | fits | 83.7–167.5 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 6.84 GiB | fits | 83.7–167.5 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 6.84 GiB | fits | 83.7–167.5 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 6.84 GiB | fits | 10.7–21.4 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 6.84 GiB | fits | 21.4–42.8 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 6.84 GiB | fits | 42.8–85.7 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 6.84 GiB | fits | 8.4–16.7 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 6.84 GiB | fits | 8.4–16.7 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 6.84 GiB | fits | 8.4–16.7 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 6.84 GiB | fits | 19.0–38.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 6.84 GiB | fits | 19.0–38.1 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 6.84 GiB | fits | 19.0–38.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 6.84 GiB | fits | 28.6–57.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 6.84 GiB | fits | 38.1–76.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 6.84 GiB | fits | 38.1–76.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 6.84 GiB | fits | 57.1–114.3 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 6.84 GiB | fits | 57.1–114.3 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 6.84 GiB | fits | 25.1–50.2 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 6.84 GiB | fits | 65.3–130.6 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 6.84 GiB | fits | 20.1–40.2 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 6.84 GiB | fits | 35.2–70.3 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 6.84 GiB | fits | 50.0–100.0 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 6.84 GiB | fits | 70.3–140.7 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 6.84 GiB | fits | 31.3–62.5 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 6.84 GiB | fits | 46.9–93.8 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 6.84 GiB | fits | 62.5–125.0 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 6.84 GiB | fits | 67.0–134.0 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 6.84 GiB | fits | 125.0–250.1 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 6.84 GiB | fits | 125.0–250.1 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 6.84 GiB | fits | 67.0–134.0 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 6.84 GiB | fits | 31.8–63.6 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 6.84 GiB | fits | 17.9–35.7 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 6.84 GiB | fits | 19.0–38.1 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 6.84 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 6.84 GiB | Needs verification | Estimated speed unavailable |
Q8_0: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 9.45 GiB | fits | 7.2–14.3 estimated tokens/s |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 9.45 GiB | fits | 8.0–15.9 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 9.45 GiB | fits | 8.0–15.9 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 9.45 GiB | fits | 14.4–28.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 9.45 GiB | fits | 14.4–28.8 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 9.45 GiB | fits | 14.4–28.8 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 9.45 GiB | fits | 21.5–43.1 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 9.45 GiB | fits | 28.8–57.5 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 9.45 GiB | fits | 28.8–57.5 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 9.45 GiB | fits | 56.2–112.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 9.45 GiB | fits | 56.2–112.4 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 9.45 GiB | fits | 56.2–112.4 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 9.45 GiB | fits | 7.2–14.3 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 9.45 GiB | fits | 14.4–28.8 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 9.45 GiB | fits | 28.8–57.5 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 9.45 GiB | fits | 5.6–11.2 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 9.45 GiB | fits | 5.6–11.2 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 9.45 GiB | fits | 5.6–11.2 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 9.45 GiB | fits | 12.8–25.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 9.45 GiB | fits | 12.8–25.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 9.45 GiB | fits | 12.8–25.6 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 9.45 GiB | fits | 19.2–38.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 9.45 GiB | fits | 25.6–51.1 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 9.45 GiB | fits | 25.6–51.1 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 9.45 GiB | fits | 38.4–76.7 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 9.45 GiB | fits | 38.4–76.7 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 9.45 GiB | fits | 16.9–33.7 estimated tokens/s |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 9.45 GiB | fits | 43.8–87.7 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 9.45 GiB | fits | 13.5–27.0 estimated tokens/s |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 9.45 GiB | fits | 23.6–47.2 estimated tokens/s |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 9.45 GiB | fits | 33.6–67.1 estimated tokens/s |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 9.45 GiB | fits | 47.2–94.4 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 9.45 GiB | fits | 21.0–42.0 estimated tokens/s |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 9.45 GiB | fits | 31.5–62.9 estimated tokens/s |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 9.45 GiB | fits | 42.0–83.9 estimated tokens/s |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 9.45 GiB | fits | 45.0–89.9 estimated tokens/s |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 9.45 GiB | fits | 83.9–167.9 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 9.45 GiB | fits | 83.9–167.9 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 9.45 GiB | fits | 45.0–89.9 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 9.45 GiB | fits | 21.4–42.7 estimated tokens/s |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 9.45 GiB | fits | 12.0–24.0 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 9.45 GiB | fits | 12.8–25.6 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 9.45 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 9.45 GiB | Needs verification | Estimated speed unavailable |
FP16: hardware comparison
| Hardware | Usable memory | Required memory | Fit estimate | Estimated generation |
|---|---|---|---|---|
| Mac mini M6 (12-core GPU) 16GB | 12.0 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| Mac mini M6 (12-core GPU) 24GB | 18.0 GiB | 16.47 GiB | borderline | 4.2–8.5 estimated tokens/s |
| Mac mini M6 (12-core GPU) 32GB | 24.0 GiB | 16.47 GiB | fits | 4.2–8.5 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 24GB | 18.0 GiB | 16.47 GiB | borderline | 7.6–15.3 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 48GB | 36.0 GiB | 16.47 GiB | fits | 7.6–15.3 estimated tokens/s |
| Mac mini M5 Pro (15-core CPU / 16-core GPU) 64GB | 48.0 GiB | 16.47 GiB | fits | 7.6–15.3 estimated tokens/s |
| Mac Studio M5 Max (32-core GPU) 36GB | 27.0 GiB | 16.47 GiB | fits | 11.5–22.9 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 64GB | 48.0 GiB | 16.47 GiB | fits | 15.3–30.6 estimated tokens/s |
| Mac Studio M5 Max (40-core GPU) 128GB | 96.0 GiB | 16.47 GiB | fits | 15.3–30.6 estimated tokens/s |
| Mac Studio M5 Ultra (30-core CPU / 64-core GPU) 96GB | 72.0 GiB | 16.47 GiB | fits | 29.9–59.7 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 256GB | 192.0 GiB | 16.47 GiB | fits | 29.9–59.7 estimated tokens/s |
| Mac Studio M5 Ultra (36-core CPU / 80-core GPU) 512GB | 384.0 GiB | 16.47 GiB | fits | 29.9–59.7 estimated tokens/s |
| MacBook Pro 14-inch M5 (10-core GPU) 24GB | 18.0 GiB | 16.47 GiB | borderline | 3.8–7.6 estimated tokens/s |
| MacBook Pro 14-inch M5 Pro (20-core GPU) 48GB | 36.0 GiB | 16.47 GiB | fits | 7.6–15.3 estimated tokens/s |
| MacBook Pro 16-inch M5 Max (40-core GPU) 128GB | 96.0 GiB | 16.47 GiB | fits | 15.3–30.6 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 16GB | 12.0 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| Mac mini (2024) M4 (10-core GPU) 24GB | 18.0 GiB | 16.47 GiB | borderline | 3.0–6.0 estimated tokens/s |
| Mac mini (2024) M4 (10-core GPU) 32GB | 24.0 GiB | 16.47 GiB | fits | 3.0–6.0 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 24GB | 18.0 GiB | 16.47 GiB | borderline | 6.8–13.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 48GB | 36.0 GiB | 16.47 GiB | fits | 6.8–13.6 estimated tokens/s |
| Mac mini (2024) M4 Pro (16-core GPU) 64GB | 48.0 GiB | 16.47 GiB | fits | 6.8–13.6 estimated tokens/s |
| Mac Studio (2025) M4 Max (32-core GPU) 36GB | 27.0 GiB | 16.47 GiB | fits | 10.2–20.4 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 64GB | 48.0 GiB | 16.47 GiB | fits | 13.6–27.2 estimated tokens/s |
| Mac Studio (2025) M4 Max (40-core GPU) 128GB | 96.0 GiB | 16.47 GiB | fits | 13.6–27.2 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (28-core CPU / 60-core GPU) 96GB | 72.0 GiB | 16.47 GiB | fits | 20.4–40.8 estimated tokens/s |
| Mac Studio (2025) M3 Ultra (32-core CPU / 80-core GPU) 256GB | 192.0 GiB | 16.47 GiB | fits | 20.4–40.8 estimated tokens/s |
| NVIDIA GeForce RTX 3060 12GB | 10.8 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 3090 24GB | 21.6 GiB | 16.47 GiB | fits | 23.3–46.6 estimated tokens/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 14.4 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 4070 12GB GDDR6X | 10.8 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 4080 16GB | 14.4 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 4090 24GB | 21.6 GiB | 16.47 GiB | fits | 25.1–50.2 estimated tokens/s |
| NVIDIA GeForce RTX 5060 Ti 16GB | 14.4 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 5070 12GB | 10.8 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 5070 Ti 16GB | 14.4 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 5080 16GB | 14.4 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| NVIDIA GeForce RTX 5090 32GB | 28.8 GiB | 16.47 GiB | fits | 44.6–89.2 estimated tokens/s |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GB | 86.4 GiB | 16.47 GiB | fits | 44.6–89.2 estimated tokens/s |
| NVIDIA RTX 6000 Ada Generation 48GB | 43.2 GiB | 16.47 GiB | fits | 23.9–47.8 estimated tokens/s |
| Intel Arc B580 12GB | 10.8 GiB | 16.47 GiB | doesn't fit | Estimated speed unavailable |
| AMD Ryzen AI Halo, Ryzen AI Max+ 395, 128GB | 89.6 GiB | 16.47 GiB | fits | 6.4–12.7 estimated tokens/s |
| NVIDIA DGX Spark 128GB / 4TB | 96.0 GiB | 16.47 GiB | fits | 6.8–13.6 estimated tokens/s |
| 2 × GeForce RTX 3090 24GB (conditional 48GB aggregate) | 40.8 GiB | 16.47 GiB | Needs verification | Estimated speed unavailable |
| 2 × GeForce RTX 5090 32GB (conditional 64GB aggregate) | 54.4 GiB | 16.47 GiB | Needs verification | Estimated speed unavailable |
A “fits” result checks estimated memory only. “Needs verification” preserves missing cache, vision, or placement information. All speed figures are estimated from the published bandwidth formula.
Field-level sources and verification dates
- id: Official source · verified 2026-10-22. Lowercase URL slug derived from the upstream repository ID; baseModel retains the exact Hugging Face identifier.
- name: Official source · verified 2026-10-22
- baseModel: Official source · verified 2026-10-22
- downloads: Official source · verified 2026-10-22
- tasks: Official source · verified 2026-10-22. Chat denotes a text-generating model, including base models that may require a chat template or instruction tuning. Code label is inferred from the repository model-family name. Vision uses pipeline metadata or vision_config.
- license: Official source · verified 2026-10-22
- commercialUse: Official source · verified 2026-10-22. Conditional commercial use: the upstream model card, section 7, identifies MIT terms and derivation from Llama 3.1 8B under the Llama 3.1 license. Review and comply with the inherited Llama terms in addition to MIT; the MIT metadata tag alone is incomplete.
- license_url: Official source · verified 2026-10-22. Pinned license file confirmed in repository listing.
- architecture: Official source · verified 2026-10-22. Classification from model_type and expert/state-space configuration fields (llama); hybrid classification takes precedence over expert count.
- parametersB: Official source · verified 2026-10-22. Exact safetensors parameter total reported by the Hugging Face API, divided by 1e9.
- activeParametersB: Official source · verified 2026-10-22. Dense model: all parameters are included. This is not a runtime FLOP measurement.
- contextLength: Official source · verified 2026-10-22. Configured maximum token positions; not a guarantee of runtime support or quality at that length. No speculative RoPE scaling applied.
- kvCache: Official source · verified 2026-10-22. Standard attention KV shape from num_hidden_layers, num_key_value_heads, and head_dim or hidden_size / num_attention_heads; FP16 cache bytes are a separate estimator choice.
- licenseStandard: Official source · verified 2026-10-22. Official standard license text supporting the permissive-license classification; model applicability is identified separately by upstream metadata.