Personally, I disabled the ghosts for all subsequent plays. I don't like the outcomes of a past game affecting my current game.
That said, perhaps there should be some smart way of placing them around dungeons?
Say, you died ten times and all your dead chars had levels between 10-40.
For the future characters, perhaps the game should keep a pool of these dead chars and randomly spawn an appropriate challenge at a relatively similar game stage.
It would need to be an extremely low chance to spawn, in practice less than 1 per game, so sometimes you don't see them at all, in order to keep a fairly "valid" pool of ghosts to spawn.
I don't know, seeing as me and probably a fair chunk of other players disabled ghosts anyway, maybe these are moot points.
As for the formal part of this RFE, I'm not sure what voting either way would aim to achieve. If I vote yes, does that mean I want them nerfed? Or that I agree they are too strong?