Feels like IMC set a population cap on the dungeons server, once the cap are reached if more ppl trying to enter they get booted.
If a group inside the dungeon finish and teleport out, the server opens up and allow another group inside.
Happened quite a few times already, if you enter immediately right after a group came out you will get inside, or if you group just finished the dungeon, you click enter right after you came out, you will often get inside.