Decision quality on JEVal
The strongest decision models approach leading thinking LLMs on single-step accuracy. SII-Decision-27B reaches 79.24% accuracy and the lowest expected calibration error (ECE) of the 25 configurations. Across domains, decision models are strongest when the evidence is available in the input; the best generative LLMs retain an advantage in medicine, finance, and law.
| Model | Accuracy ↑ | ECE ↓ | Brierraw ↓ |
|---|---|---|---|
| SII-Decision-27B | 79.24% | 0.0149 | 0.279 |
| SII-Decision-4B | 71.92% | 0.0843 | 0.383 |
| Jev | 78.38% | 0.0532 | 0.287 |
| Qwen3.8-27B · thinking | 79.49% | 0.1005 | 0.319 |
| GLM-5.3-Flash · thinking | 80.37% | 0.0784 | 0.316 |
| GPT-6-Luna · non-thinking | 80.93% | 0.1159 | 0.297 |





