RESEARCH

Model speed on one V100: latest runs

Published · Last updated

Latest read: 6 Nov 2026. We run the same open models on one V100 with the same prompts and settings, and this page shows the newest numbers and the earlier days beside them. Every number is measured by Starfield. This is not a ranking, and speed says nothing about quality.

The page is rewritten with each new read, so it always shows the latest. The sheet for each day stays public as it was read: the files are listed under Data below. A dated Model Watch post is written when something changes, such as a new model or a different version.

Latest run, 6 Nov 2026

Speed of granite-4.2-8b, Olmo-3-7B-Instruct and Qwen3-8B on one V100, run 6 Nov 2026, means of 3 repeats. All numbers measured by Starfield. Rows are in alphabetical order.
Model Output tokens per second Time to first token (ms) Output tokens per run Label
granite-4.2-8b 229.3 200.1 27,590 measured by Starfield
Olmo-3-7B-Instruct 201.8 183.3 22,533 measured by Starfield
Qwen3-8B 239.8 173.7 21,586 measured by Starfield

Output speed by day

Each cell is the mean output tokens per second of that day's repeats. The newest day is first.

Output tokens per second on one V100, 8 reads, from 28 Oct 2026 to 6 Nov 2026. All numbers measured by Starfield.
Read granite-4.2-8b Olmo-3-7B-Instruct Qwen3-8B
6 Nov 2026 229.3 201.8 239.8
5 Nov 2026 229.2 201.8 239.8
4 Nov 2026 229.2 201.8 239.7
3 Nov 2026 229.2 201.8 239.7
2 Nov 2026 229.3 201.8 239.7
30 Oct 2026 229.3 201.7 239.7
29 Oct 2026 229.3 201.8 239.7
28 Oct 2026 229.1 200.9 239.8

Over these 8 reads, output speed of each model stayed within 0.5% between its slowest and fastest day. Reads on one machine show that the method repeats. They do not show that speeds are stable in general.

Run settings

Settings of the 6 Nov 2026 run. Every earlier read on this page used the same settings.
Setting Value
Engine vllm 0.19.1
GPU Tesla V100-SXM3-32GB
Precision fp16
Prompts model-watch-prompts-v1, 150 prompts
Requests at once 8
Repeats per model 3
Pinned revision of each model in the 6 Nov 2026 run.
Model Pinned revision
granite-4.2-8b a7042144125c
Olmo-3-7B-Instruct 6e5971d9eba4
Qwen3-8B b968826d9c46

Data

The raw sheets, one per read: speed-2026-11-06.csv, speed-2026-11-05.csv, speed-2026-11-04.csv, speed-2026-11-03.csv, speed-2026-11-02.csv, speed-2026-10-30.csv, speed-2026-10-29.csv, speed-2026-10-28.csv. The same files, with the code that makes them, are in the public repository.

The full method is on the methodology page. If a number here is wrong, tell us through the corrections log.