Yesterday I used a clarity potion on myself (I had no ill effects at the time) and although the spell did not damage me as it would had I been dazed etc. I was forced to make a fort save vs negative and failing it I took a -1 con penalty. It seems odd that my perk is affecting me negatively.
Character was Geoffrey Wallace.
It's linked to the negative damage which clarity does sometimes. I'll look into it.