Benchmarks
+109%
More KV cache (2.09x). Gemma 4 12B (BF16) on RTX 5090.
+39%
More KV cache (1.39x). Qwen 3.8-27B (BF16) on RTX PRO 6000 Blackwell.
FP8 & FP4
Available for pilots. BF16 is in general release.
Models
A few models with varying architectures and sizes. Smaller .tic leaves more memory for KV cache. Published samples are BF16. FP8 and FP4 are available for pilots.
| Model | Precision | Type | Raw | .tic (savings) |
|---|---|---|---|---|
| Qwen2.5 7B Instruct on Hugging Face | BF16 | Text | 15.23 GB | 10.86 GB (28.7%) |
| Mistral 7B Instruct v0.3 on Hugging Face | BF16 | Text | 14.50 GB | 10.33 GB (28.8%) |
| Llama 3.1 8B Instruct on Hugging Face | BF16 | Text | 16.06 GB | 11.44 GB (28.8%) |
| Gemma 4 12B IT on Hugging Face | BF16 | Multimodal | 23.92 GB | 17.05 GB (28.7%) |
| Qwen3.5 27B on Hugging Face | BF16 | Multimodal | 55.56 GB | 39.57 GB (28.8%) |
| Qwen3.5 35B A3B on Hugging Face | BF16 | Multimodal, MoE | 71.90 GB | 51.27 GB (28.7%) |
| Qwen3.8 27B on Hugging Face | BF16 | Multimodal | 55.56 GB | 39.57 GB (28.8%) |
Next steps