Unsloth Dynamic 2.0 Quants
Collection
New 2.0 version of our Dynamic GGUF + Quants. Dynamic 2.0 achieves superior accuracy & SOTA quantization performance.
•
60 items
•
Updated
•
261
AI & ML interests
Open Source AI 💚
Recent Activity
View all activity
shimmyshimmer
updated
a
collection
about 10 hours ago
danielhanchen
posted
an
update
about 11 hours ago
Post
137
Mistral's new SOTA coding models Devstral 2 can now be Run locally! (25GB RAM) 🐱
We fixed the chat template, so performance should be much better now!
24B: unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF
123B: unsloth/Devstral-2-123B-Instruct-2512-GGUF
🧡Step-by-step Guide: https://docs.unsloth.ai/models/devstral-2
We fixed the chat template, so performance should be much better now!
24B: unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF
123B: unsloth/Devstral-2-123B-Instruct-2512-GGUF
🧡Step-by-step Guide: https://docs.unsloth.ai/models/devstral-2
Fixed system prompt + updates
👍
1
1
#2 opened about 21 hours ago
by
danielhanchen
Pretty bad results compared to Devstral 1 (?)
13
#3 opened 1 day ago
by
bonswouar
Fixed system prompt + Updates
👍
5
3
#5 opened 1 day ago
by
danielhanchen
danielhanchen
updated
a
model
about 22 hours ago
mmproj?
4
#1 opened 1 day ago
by
aaron-newsome
BF16 or Q8_K_XL - which would give more accurate coding results?
2
#6 opened 1 day ago
by
TimothyRoo
Pretty bad results compared to Devstral 1 (?)
13
#3 opened 1 day ago
by
bonswouar
Opencode prompting leads to "conversation roles must alternate user and assistant roles except for tool calls and results"
➕
2
15
#2 opened 1 day ago
by
Nindaleth
using it in LM Studio
1
#4 opened 1 day ago
by
Zimdin12
danielhanchen
updated
2
models
1 day ago
danielhanchen
posted
an
update
8 days ago
Post
3368
Mistral's new Ministral 3 models can now be Run & Fine-tuned locally! (16GB RAM)
Ministral 3 have vision support and the best-in-class performance for their sizes.
14B Instruct GGUF: unsloth/Ministral-3-14B-Instruct-2512-GGUF
14B Reasoning GGUF: unsloth/Ministral-3-14B-Reasoning-2512-GGUF
🐱 Step-by-step Guide: https://docs.unsloth.ai/new/ministral-3
All GGUFs, BnB, FP8 etc. variants uploads: https://huggingface.co/collections/unsloth/ministral-3
Ministral 3 have vision support and the best-in-class performance for their sizes.
14B Instruct GGUF: unsloth/Ministral-3-14B-Instruct-2512-GGUF
14B Reasoning GGUF: unsloth/Ministral-3-14B-Reasoning-2512-GGUF
🐱 Step-by-step Guide: https://docs.unsloth.ai/new/ministral-3
All GGUFs, BnB, FP8 etc. variants uploads: https://huggingface.co/collections/unsloth/ministral-3
-
- 3 replies
danielhanchen
posted
an
update
13 days ago
Post
8292
Qwen3-Next can now be Run locally! (30GB RAM)
Instruct GGUF: unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF
The models come in Thinking and Instruct versions and utilize a new architecture, allowing it to have ~10x faster inference than Qwen32B.
💜 Step-by-step Guide: https://docs.unsloth.ai/models/qwen3-next
Thinking GGUF: unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF
Instruct GGUF: unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF
The models come in Thinking and Instruct versions and utilize a new architecture, allowing it to have ~10x faster inference than Qwen32B.
💜 Step-by-step Guide: https://docs.unsloth.ai/models/qwen3-next
Thinking GGUF: unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF
danielhanchen
posted
an
update
about 1 month ago
Post
4235
You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs:
unsloth/Kimi-K2-Thinking-GGUF
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
danielhanchen
posted
an
update
4 months ago
Post
6491
Run DeepSeek-V3.1 locally on 170GB RAM with Dynamic 1-bit GGUFs!🐋
GGUFs: unsloth/DeepSeek-V3.1-GGUF
The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.
The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.
Guide: https://docs.unsloth.ai/basics/deepseek-v3.1
GGUFs: unsloth/DeepSeek-V3.1-GGUF
The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.
The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.
Guide: https://docs.unsloth.ai/basics/deepseek-v3.1
danielhanchen
posted
an
update
4 months ago
Post
5555
Run OpenAI's new gpt-oss models locally with Unsloth GGUFs! 🔥🦥
20b GGUF: unsloth/gpt-oss-20b-GGUF
120b GGUF: unsloth/gpt-oss-120b-GGUF
Model will run on 14GB RAM for 20b and 66GB for 120b.
20b GGUF: unsloth/gpt-oss-20b-GGUF
120b GGUF: unsloth/gpt-oss-120b-GGUF
Model will run on 14GB RAM for 20b and 66GB for 120b.
- 2 replies
danielhanchen
posted
an
update
5 months ago
Post
3679
It's Qwen3 week! 💜 We uploaded Dynamic 2-bit GGUFs for:
Qwen3-Coder: unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF
Qwen3-2507: unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF
So you can run them both locally!
Guides are in model cards.
Qwen3-Coder: unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF
Qwen3-2507: unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF
So you can run them both locally!
Guides are in model cards.
- 1 reply
danielhanchen
posted
an
update
5 months ago
Post
3902