Model Watch #4: a third day of V100 runs, output speed within 0.5%
We ran the same three models a third day, on 30 Oct 2026. The first two days are in Model Watch #2 (28 Oct) and Model Watch #3 (29 Oct). The settings, prompts (model-watch-prompts-v1, 150 prompts, 8 requests at once), pinned model revisions and vLLM 0.19.1 are the same as on 28 and 29 Oct. The machine is the same too: one V100, fp16, 3 repeats per model. All 450 of 450 requests finished on each of the three days. Every number is measured by Starfield.
Output tokens per second stayed within 0.5% across the three days for all three models. This is not a ranking, and speed says nothing about quality.
Results
Each cell shows the 28 Oct, 29 Oct and 30 Oct means of 3 repeats, in that order. Rows are in alphabetical order.
| Model | Output tokens per second | Time to first token (ms) | Output tokens per run | Label |
|---|---|---|---|---|
| granite-4.2-8b | 229.1 / 229.3 / 229.3 | 195.2 / 199.6 / 197.3 | 27,622 / 27,590 / 27,599 | measured by Starfield |
| Olmo-3-7B-Instruct | 200.9 / 201.8 / 201.7 | 210.3 / 183.4 / 184.3 | 22,533 / 22,530 / 22,526 | measured by Starfield |
| Qwen3-8B | 239.8 / 239.7 / 239.7 | 173.6 / 173.9 / 171.0 | 21,586 / 21,586 / 21,571 | measured by Starfield |
Output tokens per second, three days
- granite-4.2-8b28 Oct 2026229.1
- granite-4.2-8b29 Oct 2026229.3
- granite-4.2-8b30 Oct 2026229.3
- Olmo-3-7B-Instruct28 Oct 2026200.9
- Olmo-3-7B-Instruct29 Oct 2026201.8
- Olmo-3-7B-Instruct30 Oct 2026201.7
- Qwen3-8B28 Oct 2026239.8
- Qwen3-8B29 Oct 2026239.7
- Qwen3-8B30 Oct 2026239.7
How to read this
- Olmo-3's first-token mean on 28 Oct was raised by one slow first repeat, about 265 ms. It has not come back on 29 Oct or 30 Oct. The cause is still not found.
- Three days on one machine show that the method repeats. They are not proof that speeds are stable in general.
The 30 Oct files are public: the summary results, and the run files for Qwen3-8B, Olmo-3-7B-Instruct and granite-4.2-8b. The 28 Oct and 29 Oct files are linked from Model Watch #2 and Model Watch #3. The repository's corrections file is public too.
Not run yet: A100 with vLLM 0.32.0, llama.cpp, quality checks, energy use and cost.
The full method is on the methodology page. A mistake in a number goes in the corrections log.
More in Model Watch
- Model Watch #3: two days of V100 runs, output speed within 0.5%
The 28 Oct speed runs of Qwen3-8B, Olmo-3-7B-Instruct and granite-4.2-8b repeated on 29 Oct on one V100. Output speed differed by under 0.5%.