Only downside for less potential on headgear is how it looks to a buyer.
Most players buy with the assumption that they will be only using it for a while (until they find something even better) so they need potential to sell it. But, since the next person after them also will likely want to sell it, and the person after that will probably want to sell it too…
You see where it goes. For one re-sale the potential you have doesn’t matter; it can be sold. But it’s kind of like buying a car that you know will die in 20 thousand miles. It’s good for now, but, if you sell it to someone and it only has 10 thousand miles left on it, then they will be hesitant… even if they can use it for a while they’ll want to sell it to someone else too and eventually it gets to a point where the car is dead and can’t be sold. It’s just a matter of how close the buyer is to that d-day. And the closer they are the less likely they’ll think it’s a steal.
Aside from that, no, there is nothing else. Headgear is marked “unenhanceable” so there is nothing else that can be done to it (currently) aside from re-rolling an enchantment. Amulets also did not ever affect headgear in the past and I see no reason why they would when they are re-introduced in a future content release. So you’re in the clear aside from resell potential going down a little.