I suspect windows considers a network location to be new when the MAC address of the gateway changes. If the server has its network location set as public then the inbuilt firewall could be blocking your traffic.
Thanks Martin, That is exactly what happened, but it was another Peplink router that I swapped with. Why did all my devices decide this was a public ip?
Its not a public IP, the windows devices identify all new networks as a public network location until you specifically say they are private - its a security feature. I would guess that they identify a new network as being one where the gateway device physically changes.