The swordsman weakness that I find ( having played pure barb c3, pure highlander c3, pure hoplite c3, and highlander/corsair ) is the damage type weaknesses. Typically a build has you focus on a certain weapon type, and your skills ( most of your DPS ) are attached to one weapon type.
Slash is the easiest type to get for a swordsman, and it’s the worst damage type. Plate mobs are high def high HP and annoying to take care of, and you deal less damage to them with slash. Cloth mobs are low HP low def and easy to defeat even with opposing weapons, and you deal bonus damage to them - great, so what. Ideally you’d be able to swap weapons and have that weapon’s damage type be reflected in your skill, unless it’s a very specific skill. Barbarians are a bit better off here, as they can use the Naktis 2h sword ( Strike ), and their damage does come more from auto attacks.
You also can’t do much about Ghost-types reducing your damage.
I have also played a cryo3, and found that I had a much easier time dealing with damage types. They have access to magical ice/pierce, pierce being the best damage type ( weak against cloth but you’ll still one-shot them ). Nothing is strong against magic, and one type is weak against magic. So you’re not dealing with certain mobs that are just hard to kill for your class.
Priest c2/c3 is good as well, with your buffs and the access to any weapon type + arde dagger work very well. You can use strike/pierce/slash as you please, and still do a lot of bonus damage.