Avantae is irrelevant to this since he lives close to the paradise. Can you imagine flying from Cali and back and then fly again for rivals? Tae only has to travel to the rivals camp.
He said that he tried to come to the paradise camp but hes going to a camp this week (I forgot where exactly) and it would be way too much traveling for him.