Very mild spoilers ahead...
I think these factions' member NPCs are not appropriately flagged as "evil". This is certainly true of the Breadbloods, active necromancers. I think certain types ARE evil, though. It's the regular grunts that don't seem to be.
But really, both are murderous squatters. I think this is a bug,