Firmware 7.0 and IPSec VPN issues


#1

Upgraded from 6.3 to 7.0 beta firmware.

I have two IPSEC VPN tunnels that were created with the 6.3 firmware to Microsoft Azure. Tunnels worked without issue. After upgrading to 7.0 I am seeing a couple of issues.

1st - The VPN tunnels bounce between three states, established, partially established, and connecting. When it is established or partially established - connectivity remains. When it enters a connecting state the tunnel is down and packets are dropped. This seems to happen for 5-10 seconds at a time. It never did this with the 6.3 firmware. The tunnel already remained connected. I’m using policy based VPN routing with Azure using IKEv1 (which is supported with Peplink).

2nd - The CPU is extremely high, hitting 100% frequently. I disabled both VPN tunnels as a test and the CPU returned to normal. Don’t know if the CPU is spiking because of the 1st issue? At present I have the tunnels disabled and connecting to my tenants via the Internet because the CPU spikes are causing performance issues on the router.

The hardware is a Balance 20. As noted, the VPNs were created over a year ago and never had any issues with 6.3.

It should be noted that my AP issues were resolved due to Firmware 7.0 (happy) but now I have these VPN issues.

Link to diagnostic report…
https://drive.google.com/file/d/0B0chNgGbWQS4N3IwNEQxdjFQME0/view?usp=sharing

Thanks,
Ryan


#2

Please help to turn on Remote Assistance for further checking.

Thanks.


#3

Remote Assistance is enabled.


#4

@Ryan_Finger, we failed to access the Balance 20 via Remote Assistance. Please help to disable and re-enable it. If there is a firewall in front of Balance 20, please ensure ra.peplink.com is reachable by Balance 20 (you may test via System > Ping) and outbound TCP 80 and 443 were open on the firewall.

Thanks.


#5

I also noticed the CPU spikes on my balance 30 when managing APs and using VLans.

One thing to check is that your outbound policies aren’t moving your tunnels from one link to another. I imagine the tunnel initiator IP repeatedly changing may cause some issues with Azure. It could also be service interruptions in Azure itself. Are you having any other issues with any other type of persisted connections?


#6

Issue occurred immediately after I upgraded from 6.3 to 7.0. I have two tenants, both in different regions and both with their own IPSec tunnels back to unique gateways in Azure. Both are doing the exact same thing. I tried to disable one of the VPN tunnels, which did reduce the CPU load a bit but it is still too high. Disabling both of the VPN tunnels returns the CPU load to “normal.” (Based upon what I saw previously with 6.3).


#7

Remote assistance is disabled and re-enabled. There is no firewall in front of the Balance 20. ra.peplink.com is reachable. No firewall rules should be in place.


#8

FYI… I have disabled and re-enabled remote assistance.

Thanks,
Ryan


#9

@Ryan_Finger, can help to open ticket? This allowed us to follow up easily.


#10

I had this same issue immediately after updating to 7.0. In my case, I had two ipsec VPN tunnels on a Balance 305. Only one of the VPN tunnels kept dropping and reconnecting every few seconds and behaving similar to that described above. In my case, swapping the wan connection used for the bad VPN only, resolved the issue. I did not try further diagnosing the cause or swap them back.


#11

This is an interesting finding. It seems like WAN link issue. Possible to revert the settings and observe whether the similar problem happens on the bad VPN again? If the problem persists, please downgrade to previous firmware version immediately. The problem solved immediately?

This allowed us to isolate the problem. Thanks.


#12

I am having the same issue as above with a Balance One after upgrading to version 7. I had an IPSec VPN to a Cisco ASA 5505. Even after re-creating the VPN, I can’t get the tunnel up. I am willing to help diagnose this issue. BTW, I upgraded from the 2715 build to the 2742 build in hopes that it would fix the issue, so I can’t downgrade now.


#13

I have since added a third VPN tunnel as I am migrating them over from other hardware, but to help you troubleshoot, I will move the “bad” one back and see if I still have the problem, though I doubt not. I do it now and report back after some hours unless it is immediately unstable.


#14

It seems that I spoke too soon. I logged in to make the switch and the VPN connections are going from green to red over and over, each time, this triggers an alert which I had turned off because it was annoying to get hundreds of emails.

Interesting is that I am not actually not losing the ability to ping across the VPN, but the event log has several connection and disconnection logged several times per second.


#15

@maverill, please upgrade to this firmware - http://download.peplink.com/firmware/plb1/fw-b1_210hw4_310hw4_br1ent_transit_m700hw3_hd2mini_hd4-7.0.0s012-build2767.bin

@theMRGABE, thanks for your efforts. Please upgrade to one of the firmware below:
Balance 305 Hw2 - http://download.peplink.com/firmware/plb2500/fw-b305hw2_380hw6_580hw2_710hw3_1350hw2_2500-7.0.0s012-build1928.bin

Thanks.


#16

Build 2767 fixed my issue. Thanks for the quick response.


#17

I am same issue. I have VPN setup between Balance 210 and Balance 20. After upgrading both to 7.0, VPN broke. I checked the log with Balance 20 and it says “IKE/ESP Proposal refused, please verify settings”. So I double checked and even changed the setting on both sides but it didn’t work. I see this thread about build 2767 so I upgraded Balance 210 with this firmware to give it a try and it didn’t fix the issue.


#18

@simon, please open ticket for us to investigate.

Thanks.


#19

New Peplink Balance 580 HW 3 with the same ipsec VPN connection issue for Firmware 7.0.0 build 1904

if i boot to the old firmware 6.3.1 build 1631 it connects fine with the same configuration.


#20

As mentioned here, please open ticket.

Thanks.