That is very weird then. Here’s a video of another test I made a few minutes ago, and a screenshot of the weapon I was using.
My mATK = 712
Wendigo Searcher mDEF = 192
Weapon = +1 Fedimian Club with 1 star blue gem (+5 mATK)
Sub-weapon = Stock Arde Dagger (153 Fire Damage, Wendigo Searcher is Fire type, damage reduced to 50% or 76.5)
OOB Normal Damage = 3333, or 999
OOB Buffed Damage = 6863+2, or 2060
If we go by my formula:
Normal OOB Calculation with Maces
((mATK + 70 - mDEF) + Elemental Damage) * 1.5
((712 + 70 - 192) + 76.5) * 1.5 = 999
Buffed OOB Calculation with Maces
(((((mATK - Gem mATK) * 2) + 70 - mDEF) + Elemental Damage) * 1.5) + (Gem mATK * 1.5)
(((((712 - 5) * 2) + 70 - 192) + 76.5) * 1.5) + (5 * 1.5) = 2060
Just a quick edit, I realized I can just subtract the Gem mATK * 1.5 all together like this:
((((mATK * 2) + 70 - mDEF) + Elemental Damage) * 1.5) - (Gem mATK * 1.5)
And now that I think about it, subtracting the gem damage doesn’t make much sense, even if the result is the same, I believe your formula is the one the devs intended.