Guides

Benchmarks

Measure model speed and quality instead of relying on gut feeling.

Speed test

Uses llama-bench (the official llama.cpp tool) with several repetitions. Reports prompt processing and generation in tokens/s. It stops the running server for the duration to free VRAM, and warns if GPU clocks were not at maximum.

Quality test

A set of 18 tasks (math, logic, code, format, knowledge, precision) graded automatically and deterministically — no AI involved. The result includes reasoning stability: the share of answers that finished naturally rather than being cut off at the token limit.

Perplexity

Computed over the full wikitext-2 test split, the same corpus llama.cpp's own examples use, so it is comparable to published numbers. Lower is better.

Comparing models

Add several models and compare them side by side. “Quick compare” runs your own prompt on each; an AI judge can optionally score it (this is not an objective benchmark).