Now you’ll have to suffer 8s cast time with Rune Caster 
Better replace RC with Wiz3 for Quick Cast attribute 
The only area i would be suffering though, is the warlock abilities no? the rest of the other DPS abilities are channeled. or does casting speed affect the speed of the channel? but that dosn’t really matter anyway does it seeing as the cd will be the same after the channel anyway.
And is the cast time that detrimental given surespell? pop surespell / cc / high cast time abilities and then followup with your other spells
You’re missing the point.
You have a choice between adding 50% dmg to your PsychoWarlock skills (+ Magic Missile) or adding some 8s CT, big ACD skills.
First is better.
Don’t lose yourself in mismatched class combinations. Keep it simple, logic.
Would it be viable to switch out Warlock for RC?, looking at the play styles i think i like RC better, but i can’t find any reliable information on them. apart from a few youtube videos and badly worded descriptions.