SpeedFusion reducing bandwith and increasing latency

malden · January 28, 2021, 12:05am

Hi,

I bought a Balance310x and am running the following configuration:

2 x ISP-Uplinks <-> Balance 310x <-> Sophos UTM Firewall

Uplinks are
1 x VDSL 100/25 Mbit (idle 10ms)
1 x Cable 500/50 Mbit but only delivering 300/20Mbit/s (idle latency arround 22ms)

The Cable Uplink is configured as drop-in mode with the firewall
VDSL uplink is in NAT mode.

NAT on the Sophos Firewall is active.

A speedfusion VPN with 5 sub tunnels is configured to a speed fusion hub provider for testing purpose. All 5 tunnels are configured with Cut-off Latency 150ms and Suspension Time after Packet Loss 100ms on both sides.

I testet all 5 subtunnels by running speedtests and packet loss tests (onboard speedtest on the Peplink and browser speedtest on client)

Here are my first test with Bonding_vpn tunnel:
The speedtests are showing strongly fluctuating results between 80-150 Mbit/s for download and 18-22 Mbit/s for upload. I would have expect a much better bandwith in bonding mode. (total sum of boths uplinks minus 25% overhead - > 300/35 Mbit/s.)

Download: While running the speedtest the monitored latency for every uplink (via Status/speedfusion) while downloading is not exceeding 40ms and there is no packet loss shown.
I am wondering what causes the peplink to limit the bandwith when no packet loss and no latency exceeding the cut-off limit of 150ms is occuring for every single uplink of the tunnel ? Why is the resulting bandwith of the bonding so low?

Upload: While testing the upload speed the latency of the uplinks vent up to 500-1000ms and there is packet loss some times.

I’m wondering why the cut off mechanism is allowing a throughput that is is causing such high latencies on the uplinks.

Any ideas?

Regards
Malden

WillJones · February 1, 2021, 12:01pm

Hello,

You have quite a difference in potential bandwidth between your two WAN links (at least downstream), such large differences when working at relatively high speeds can be hard for the bonding algorithm to cope with in my experience.

You say you measured using the built in bandwidth test on the Peplink, was that the WAN analysis tool or the PepVPN tester, the former allows you to test bandwidth between the 310X and your FH outside of the Speedfusion VPN and could show you if there is some capacity limit between you and the FH.

When you load both lines with traffic do you also see any significant number of out of order, fragmented or retransmitted packets? You can monitor for this on the graphs on the SF status page either on the hub or the 310X. A very high number of fragmented packets could indicate an MTU problem.

What are your different sub tunnels configured for too, frankly for the purposes of testing I’d strip the config back to the bare minimum to make testing easier. Also have you tried dynamic weighted bonding (configured via the support page) although intended for cellular networks I’ve had good results using it in situations such as yours with large differences in WAN speeds.

malden · February 4, 2021, 10:42am

Hi WillJones,
thanks for your reply, much appreciated.

For testing I used both, a speed tester browser based and the peopling speedtest under Status->SpeedFusion->“PepVPN Test Configuration”. For the mentioned WAN analysis, tool to the FusionHub: Is there a special test config necessary on the hub for which I have to ask the hub provider? Just using the IP of the hub doesn’t work for me.

To answer your second question, I attached a screen shots showing the graphs while one user was uploading some stuff to Dropbox over the WAN-smoothing-VPN tunnel.

I added the monitoring via ping plotter from a client during the same time as well.

The two attached screenshots of the PepVPN tester are showing the results and the retransmitted packets during this Drobboy uplad.

I have 1 tunnel with 5 sub tunnels configured:

Bonding
WAN smoothing (normal) / Forward error correction low,
Balanced /Forward error correction off
LowLat / Forward error correction off
DynamicWeighted Bonding all five tunnels: cutoff latency 150ms/ suspension time 100 ms)

DynamicWeightedBonding is delivering the best total bandwidth (300Mbit/s down and 32 Mbit/s up) but is resulting in some high latencies. In the evening hours those latencies are getting much higher than in the morning ours (general usage of the internet on provider side)

What I just realised while testing is, that when I’m testing the Dynamic-Weighted-Bonding tunnel via the PepVPN Tester (up and download) that a client which is using the WAN-Smoothing tunnel doesn’t see noteworthy increasing latency (<50ms) but a client which is using the same WAN-smoothing tunnel is seeing latencies >400ms.

So looks like it would make sense to put the clients with higher bandwith requirements to the DWB tunnel and those which requires low latency on the low latency tunnel.