I think this is a reasonable idea. Not sure if necessary, but if this is enacted, it shouldn't be made too draconian. A single line of text would be fine.
It's obviously nice to have some decent feedback on your RFE, especially from those voting No, but I don't think limiting votes to only the long-winded is a good idea, either.
I would be tempted to upvote in support except for two things: one, sometimes people simply don't have the time or confidence to converse but want to vote on an issue they care about, and two, I completely disagree with the idea of forcing user votes to be publically visible - this opens people up to far too much potential retribution. They're not the sole decisive factor when RFEs are approved or rejected so they shouldn't need to carry that much weight.
So that's my two cents, no vote cast.