L2TP VPN issue

Hello, I have multiple users connecting using the L2TP connection. They are able to connect when trying from different remote locations. However, when two or more are working at the same remote location it only allows one person to connect at a time. When the second person connects the first person gets dropped.
What do I need to do to fix the above problem?