We had this problem with our scrambler a couple years ago and our Electric vehicle last year. By putting the brake (where the wingnut stops) on one side of the car, it stops that wheel sooner, because their is some flex in the axle. This year we are putting our brake in the middle of the car to fix that problem. Back on topic, I agree that It should go in a straight line, but I think you should be able to tap the wall without penalty.
I serously doubt that the cause of your vehicle turning during braking is due to the uneven toursional flexing of the axle. A much more likely cause is that the wheels are locking up completely and causing the tires to skid. Once your wheels are no longer turning they lose all directional control. The fact that the vehicle always turns in the same direction when the brakes are applied is probably simply due to the placement of the CG or the uneven traction (due to tire compound or weight distribution) of the tires. It is extremely important to devise a braking system that does not cause the wheels to skid if you want predictable direction and stopping distances. The wingnut system can work on a slow scrambler but not on a competitive one.
We had the fastest scrambler by far at Nationals last spring, we were getting travel times around 2 seconds, but our stopping distance was extremely surface dependent. We didn't have sufficient time to work out a really good solution so we found a way to get the brakes to apply slowly and it helped greatly. Unfortunately what we didn't plan for is the fact that the vehicle accellerated so rapidly that the back of the egg actually cracked without hitting anything outside of the scrambler. Our normal strategy would have been to use the first run to figure out stopping distance based on the track surface and then set the second run to just hit the wall as the vehicle stopped. But since the back of the egg was already cracked we had to be extremely conservative on our second run to be absolutely sure that the egg didn't touch the wall and break the egg. In the end I think we were 10 cm short of the wall.
Interestingly enough the fact that the vehicle went so fast meant that we had to very carefully adjust it to go straight. If the vehicle had even the slightest turn it would skid and shoot off the track as soon as it left the launcher, sometimes it would even roll over from the G forces of the turn. We got all that worked out at the last minute but we weren't prepared for a launching G force induced crack in the egg.