Check your CPU load during those moments (and RAM data transfer rate; I get considerably better framerates than a friend of mine who has a computer that’s almost identical to mine (same MB, same CPU, same graphics card (including vendor/clock speeds, same RAM amount, different clock/timings)
Edit:
To be accurate:
The both of us run:
MB: ASUS Z97M PLUS/BR
CPU: i5-4460 (stock)
Graphics: EVGA GTX 970 (Reference)
I run 2x8GB Kingston HyperX Fury 1866MHz CL10 RAM, he runs 2x8GB Corsair Value 1600MHz CL11
Software setups are also pretty similar (in fact, I multitask more than him while gaming).