Reports

Measurements of spark on real machines: how fast it answers, and how often it is right. Each report comes with its whole dataset.

One developer's experiment, shared as it is. Not a promise of performance: another machine, another model or another day gives other numbers. MIT licence, no warranty.

• spark v1.102 on two machines and every model auto may pick
  2026-10-27, spark v1.102. Two machines, nine models on the mini-PC and two on the MacBook Air, 6,571 graded answers. gemma4-26b-a4b still leads. Every model auto may pick now has a grounding score.
• spark v1.87 on two machines and two models
  2026-10-05, spark v1.87. Two machines, two Gemma 4 models, 1,102 graded answers to the first report's questions and 576 to new ones. gemma4-26b-a4b meets the bar on both machines.
• spark v1.85 on two machines and two models
  2026-10-04, spark v1.85. Two machines, two Gemma 4 models, 1,102 graded prompt-line answers. No run meets the bar. spark's knowledge of its own verbs is the weak spot.