It seems like maybe you’re using a wind instrument? If you wanted to counteract this, you could choose a style of instrument that doesn’t vary in pitch all that much depending on how you initiate the note (eg. percussion).Two cents (literally two things):
I feel like the average is quite possibly the WORST way to judge the pitch of an instrument. If the player starts off flat (or very flat) but is able to correct the pitch, it doesn't matter. The average is not a resistant measure of the center. Median would be better, but still, not great. Why not stick with the best? It's perfectly fine as it is and gives the player an opportunity to recover.
Building on that, I strongly believe those with perfect pitch have a strong inherent advantage over those without perfect pitch. It's my very strong opinion (and recommendation), that each note (obviously from the tuner) be played for about 3 seconds prior to the player actually playing the note. It levels the playing field better and gives those that aren't blessed with perfect pitch (like me) a more fair chance at getting a better pitch score.
1. Taking the average is terrible or any measure in that sense; the best is still the best
2. Play the in tune pitch before each note to mitigate the advantage of perfect pitch
Also, I agree that average pitch is not a great method of measuring pitch, due to a whole host of aforementioned issues, but best pitch doesn’t really work either. Even if your instrument is wildly out of tune if it either doesn’t hold steady or is adjustable on the fly, it’s possible to pass through the correct pitch briefly, which seems kind of unfair compared to the instruments that can be carefully tuned to hit the correct pitch steadily (that seems like it should get a higher score than one that just so happened to hit the correct pitch, at least to me).