After experimentation, I am quite certain that the number of HP you gain on a level-up depends on your current toughness score, including bonuses from items. That means you can improve your HP, if you switch your amulet to the elemental Earth shard just before levelling up. (The shard of elemental fire is a better choice to use the rest of the time, imo).

Is this a desirable feature or not? I'd estimate that doing this improves your final HP score by some 200 points, so it's not a trivial matter.