Invisibility Only Works Under Certain Conditions
Monsters continue to know where you are after you become invisible.
If you are invisible when you first come within view of a monster, you can beat the monster to death without problems as long as you stay invisible for the entire time. The monster will not run, just wander around until you kill it, even if it is attacked several times. This seems unbalanced, but that is for another RFE.
If, however, you are visible and a monster becomes aware of you, and you become invisible later, the monster will still know where you are at all times, and attack you.
To reproduce: come within view of a monster while visible. The monster will start to give chase. Now equip a means to become invisible, such as a ring of invisibility. The monster should still be chasing you, even though it can no longer see you.