[SPOILER] XXX's pathfinding is wonky in some locations
Since Yggaz is coded to not approach regions of Terinyo, you have to do a lot of swapping to get him out. Area limits seem to apply to other locations aside of Terinyo. This makes getting him from place to place a chore.
Swapping is made slightly easier by the fact that he compltely stops when the PC is unreachable (meaning he has no way to approach the PC). If it's not actually intended to make swapping easier, it's possibly a weird/buggy behavior.