If for any reason the relic leaves your inventory, even though you remain attuned to it, you can't equip it without taking the extra step of re-attuning. This isn't often a problem except in the case of weapon relics which can be disarmed and thus require an extra step to re-attune before they can be re-equipped which is a real bother and could use a fix.
Still a huge problem and if for any reason your relic leaves your inventory even if you get it back and are still attuned to it, you can no longer cast spells until you attune again. Please fix this or change it so relics when disarmed only go into your inventory.