Sampling temperature changes what you are measuring

Generation is sampled, and the settings that govern sampling decide how much of the output is determined by the request and how much by chance. Run the same prompt through the same candidate ten times at a high temperature and you will collect a spread of answers; that spread is the noise floor of your test. Any gap between candidates smaller than it is not a finding.

Measure the noise floor first

Pick a handful of representative examples, rerun each five or ten times against a single candidate, and score every run. What you get back is the run-to-run variation you must beat before a between-candidate difference means anything. It takes an hour and it retires most arguments about small differences before they start.

Fix the settings, and report them

Sampling settings belong in the record alongside the prompt and the date. Lowering temperature narrows the spread but does not eliminate it, and it changes the task: a comparison at near-zero temperature measures the most likely output, which is the right target for extraction and classification and the wrong one for anything where variety is the point. Choose the setting that matches the deployment, keep it identical across candidates, and write it down.