Model Watch #3: three days of V100 runs, output speed within 0.5%

Updated 30 Oct 2026: added the 30 Oct runs as a third day. The 28 Oct and 29 Oct numbers are unchanged.

We repeated the speed runs from post 2 on 29 Oct 2026 and again on 30 Oct 2026. The settings, prompts (model-watch-prompts-v1, 150 prompts, 8 requests at once), pinned model revisions and vLLM 0.19.1 are the same. The machine is the same too: one V100, fp16, 3 repeats per model. All 450 of 450 requests finished on each of the three days. Every number is measured by Starfield.

Output tokens per second stayed within 0.5% across the three days for all three models. This is not a ranking, and speed says nothing about quality.

Results

Each cell shows the 28 Oct, 29 Oct and 30 Oct means of 3 repeats, in that order. Rows are in alphabetical order.

How to read this

• Olmo-3's first-token mean on 28 Oct was raised by one slow first repeat, about 265 ms. It did not come back on 29 Oct or 30 Oct. The cause is still not found.
• Three days on one machine show that the method repeats. They are not proof that speeds are stable in general.

The 30 Oct files are public: the summary results, and the run files for Qwen3-8B, Olmo-3-7B-Instruct and granite-4.2-8b. For 29 Oct, see the summary results and the run files for Qwen3-8B, Olmo-3-7B-Instruct and granite-4.2-8b. For 28 Oct, see the summary results and the findings note. The repository's corrections file is public too.

Not run yet: A100 with vLLM 0.32.0, llama.cpp, quality checks, energy use and cost.

The full method is on the methodology page. A mistake in a number goes in the corrections log.