An alternative approach might be to make it so that neutrals and lawfuls facing co-aligned hostile monsters can pray to have said monsters calmed. This could cost some piety (not proportional to number of monsters to be calmed) for all except for champions of Law/Balance, who would be able to do it for no piety cost. And the god would be impotent to do it against uniques (so if you attack, say, the Little Girl, or anger Dwarftown, or come upon a non-chaotic random artifact guardian, don't expect your god to undo the damage or make it easy for you).
To avoid accidental calmings, perhaps you can only do it while at least one such hostile monster is visible.