As near as I can tell from the combat log, Doom is bugged. It is supposed to give a damage penalty of 2 points. This is done by giving -2 magical damage, it seems, but if you are not doing magical damage the penalty is reduced to zero.
From what I understand, while it does appear this way, It does actually work.
Its weird, I know but since the spell cant preempt what sort of damage the player is going to cause at any given moment this is the way it was implimented by the devs.
I'm pretty sure if you actually check the damage numbers, it does function.