We had a similar problem with one glider. We played with flaps for awhile, but did not find the issue. Finally we put a different wing on the fuselage (taped on), and it did the same thing, while that wing was fine on its own fuse. Therefore, we cut the stab off and put on a new stab and fin, problem gone.
Whatever was not straight was subtle. Small errors on the tail group can cause big issues.
I would certainly look at flaps for differential flex, and look for wing warp. But if you have more than one plane (you should), try mix/match parts for diagnostic reasons.
Coach, Albuquerque Area Home Schoolers Flying Events
2016 C WS 8th place
2018 B WS 2nd place
2018 C Heli Champion
2019 B ELG 3rd place
2019 C WS Champion
AMA Results: 3 AAHS members qualify for US Jr Team in F1D, 4 new youth senior records