FYI, I also found a fragment at the top level of the tower.
Just wondering... right now all these fragments are immune to destruction because they're artifacts, but this has the side effect of accelerating the artifact count in the game. Is there another way to protect them from destruction, that doesn't change the precrowning/postcrowning calculus?