Ternary Bonsai 2
Qwen models compressed to ternary weights: a 27B-class model in 5.95 GB and a 2B model in 0.43 GB, all with vision.
In short. Bonsai 2 takes popular Qwen models and squeezes each weight down to one of just three values, which shrinks the files to roughly a ninth of their normal size. The 27B version fits in about 6 GB, so it runs on a laptop, and the 2B version needs under half a gigabyte. All four sizes understand images and think before they answer.
Summary
Ternary Bonsai 2 is a family of four vision-language models from PrismML, a company in Pasadena, California. They are not trained from scratch. Each one is derived from an Alibaba Qwen model with the architecture unchanged: Qwen3.8-27B for the largest, and Qwen3.5-9B, 4B and 2B for the others. PrismML's contribution is the compression. Every language-model weight, including the embeddings and the output layer, is stored as −1, 0 or +1, with one 16-bit scale for each group of 128 weights, which comes to about 1.7 bits per weight instead of 16. The 27B model was announced on 17 September 2026 and the 9B, 4B and 2B models on 15 October 2026. All four use Apache 2.0, the same license as their base models.
The practical effect is file size. The 27B language model is 5.95 GB in the densest GGUF packing, called PTQ1_0, against about 54 GB at 16-bit, and a 7.21 GB packing called PQ2_0 decodes faster on some GPUs. The 9B, 4B and 2B files are 2.00, 0.95 and 0.43 GB. Image input needs a separate vision-projector file of 0.36 to 0.63 GB, which is loaded only when an image arrives. All four sizes keep the base model's 262K-token context, call tools, and think before they answer by default. A 2-bit MLX version for Apple silicon is published too.
Quality, as reported by the developer. On PrismML's own 14-benchmark suite, run in thinking mode with EvalScope and vLLM on an H100, the 27B model averages 84.78, which is 98.2% of the 86.32 scored by the 16-bit Qwen3.8-27B. A conventional 2-bit build of the same base model (IQ2_XXS, 7.27 GB) averages 72.59 on that suite. Math (96.57 against 97.06), coding (89.42 against 89.07) and tool calling (BFCL v3, 74.92 against 76.74) stay close to the baseline. The losses are in knowledge and reasoning (79.86 against 85.55) and vision (66.19 against 71.36). For the three smaller sizes, PrismML's launch post compares 19 benchmarks with the matching Qwen3.5 models. The 9B keeps about 92% to 100% of the baseline score in each of six skill areas, and its weakest rows are Tau2Bench (73.02 against 85.25), GPQA Diamond (69.9 against 82.53) and the Omni_doc document benchmark (73.8 against 88.73). The 2B falls to 39.6 on GPQA Diamond, where Qwen3.5-2B scores 58.48. These are PrismML's own results and we have not reproduced them.
PrismML's llama.cpp fork runs the two compact packings on NVIDIA GPUs, Apple silicon and CPUs. A third file type, Q2_0, is a little larger (7.63 GB for the 27B) and runs on recent builds of upstream llama.cpp. Older builds load it without a warning and produce garbage, so check your build first. PrismML reports about 130 tokens per second for the 27B on an RTX 5090 and about 28 on an Apple M5 Pro laptop, measured with the PQ2_0 file. The launch post for the smaller models adds that Bonsai 2 delivers up to 5 times the throughput and 3 times the energy efficiency of the matching full-precision models on hardware such as an RTX 4090 and an M5 Pro. Use the sampling settings on the model cards, which differ between thinking and non-thinking modes. The default reasoning effort is xhigh, so give the model a generous output limit. The low setting is not supported and behaves close to xhigh.
The first-generation Bonsai models, 1-bit and 1.58-bit versions at 27B, 8B, 4B and 1.7B, remain on Hugging Face but are superseded. This entry covers Bonsai 2 only.
Good for
- Running a 27B-class reasoning model with image input on a laptop or a single consumer GPU
- Phones, tablets and always-on background tasks, where the 4B (0.95 GB) and 2B (0.43 GB) sizes fit
- Private or offline assistants that keep prompts and files on the device
- Commercial products, since all four sizes use Apache 2.0
Watch out for
- The two compact packings need PrismML's llama.cpp fork or its MLX build, while stock llama.cpp runs only the larger Q2_0 file and only on recent builds
- The compression costs some quality, mostly on knowledge, reasoning and vision tasks, and more at the small sizes
- The models think by default and need a large output limit, or answers are cut off mid-thought
- The densest 5.95 GB file decodes more slowly than the 7.21 GB one on H100, A100 and Blackwell cards
- All benchmark and speed numbers are PrismML's own and have not been independently checked
Performance claims come from the developer and have not been independently verified.
How much memory does it need?
Estimates for the Ternary Bonsai 2 2B variant. A mixture-of-experts model must hold all of its experts in memory even though only some are used for each token, so total parameters decide the memory and active parameters decide the speed.
| Variants | Parameters | BF16 | FP8 / INT8 | 4-bit (Q4) | Fits on |
|---|---|---|---|---|---|
| Ternary Bonsai 2 2B TERNARY | 2.21B | 5.08 GB | 2.54 GB | 0.473 GB | Laptop or phone |
| Ternary Bonsai 2 4B TERNARY | 4.54B | 10 GB | 5.22 GB | 1.04 GB | Laptop or phone |
| Ternary Bonsai 2 9B TERNARY | 9.41B | 22 GB | 11 GB | 2.2 GB | Laptop or phone |
| Ternary Bonsai 2 27B TERNARY | 27.4B | 63 GB | 31 GB | 6.55 GB | Laptop or phone |
Estimates: weights plus about 15% overhead. Real usage grows with context length and depends on the software.
The license in plain words
One of the most widely used permissive licenses. You may use, change and sell products built on the model with only light conditions.
See also
- Best open-source multimodal models in 2026 (images, video and audio)
- Best small open models you can run on a laptop
- PrismML's Ternary Bonsai 2: a 27B-class model in 5.95 GB
Frequently asked questions
- What is Ternary Bonsai 2?
- Bonsai 2 takes popular Qwen models and squeezes each weight down to one of just three values, which shrinks the files to roughly a ninth of their normal size. The 27B version fits in about 6 GB, so it runs on a laptop, and the 2B version needs under half a gigabyte. All four sizes understand images and think before they answer.
- Can I use Ternary Bonsai 2 commercially?
- Yes. Ternary Bonsai 2 is released under Apache 2.0, a permissive license. You can use it commercially if you follow the standard conditions, such as keeping the license notice.
- Is Ternary Bonsai 2 open source?
- Ternary Bonsai 2 is released under Apache 2.0, a license approved by the Open Source Initiative. Training data and training code may not be public, so it is best described as open weights under an open-source license.
- Can I run Ternary Bonsai 2 on my own computer?
- Yes. The smallest variant, Ternary Bonsai 2 2B, needs about 0.473 GB of memory at 4-bit precision, which fits a modern laptop or a mid-range graphics card.
- When was Ternary Bonsai 2 released?
- Ternary Bonsai 2 was released by PrismML on September 17, 2026.
- How long a context window does Ternary Bonsai 2 have?
- Ternary Bonsai 2 supports a context window of up to 256K tokens.
Official links
Sources
Specifications come from these primary sources. Benchmark claims, where mentioned, are reported by the developer.
- PrismML: PrismML Launches Bonsai 2 27B, Its Most Capable Model Yet (17 Sep 2026) prismml.com
- PrismML: Introducing Ternary Bonsai 2 2B, 4B, and 9B (15 Oct 2026) prismml.com
- Ternary-Bonsai-2-27B-gguf model card (Hugging Face) huggingface.co
- Ternary-Bonsai-2-9B-gguf model card (Hugging Face) huggingface.co
- PrismML documentation: Ternary Bonsai 2 27B docs.prismml.com
- Bonsai 2 27B whitepaper (PrismML) github.com
Release date and facts last verified November 11, 2026.
Related models
| Model | Released | Parameters | Context | License | Smallest variant needs |
|---|---|---|---|---|---|
| Qwen3.8-27B Alibaba Qwen | 27.8B | 256K | Apache 2.0 (Permissive) | 19 GB Gaming GPU | |
| Qwen3.5 Alibaba Qwen | 873M – 403B 17B active | 256K | Apache 2.0 (Permissive) | 0.602 GB Laptop or phone | |
| Dory 1.0 NVIDIA | 4B | 256K | OpenMDW-1.1 (Permissive) | 2.76 GB Laptop or phone | |
| Gemma 4 Google DeepMind | 5.1B – 30.7B | 256K | Apache 2.0 (Permissive) | 3.52 GB Laptop or phone |