It would be cool if Improved Unarmed Strike gave +2 bludgeoning damage when wielding a holdable tool.
It would also be cool as a Bard/Rogue minor perk.
I don't dislike it, but not really worth the trouble of implementing that, I'm afraid.
Not sure if the holdable items can have the relevant item properties, but would be easier to just slap extra damage on holdable items and reward them to cool PCs.