Hi @blade, this is impressive, you have done a lot of tests here!
I’d suggest you to upgrade to firmware 8.1.0 RC 1, because the new SpeedFusion chart will show a lot more information, and also we have a new SpeedFusion algorithm “Dynamic Weighted Bonding” which is designed for Cellular connections and may perform differently compare to the default bonding algo:
==
Also, for me to get a better understanding of your environment the best way is by enabling Remote Assistance (RA):
With RA enabled, Peplink team (including me) will be able to access your device remotely and do some tests and apply configuration change to it. Is this possible for your case? After enabling RA, you can either send me the Serial Number using direct message (don’t post it in public forum post), or create support ticket here then I’ll follow up: https://ticket.peplink.com/ticket/new/public
==
And if possible, I’d like to see how it perform when you test with WAN Analysis tool using one Cellular WAN at a time, and both Cellular WAN simultaneously, test mode using TCP is fine. Looks like both LTE is on the same ISP, and and it’s possible a single Cellular already consumed all the available bandwidth on the cell tower side and thus using both WAN together doesn’t give any benefits.
It seems that the speed parameters were ignored and test was done at a lower speed.
These LTEs are not from the same ISP. These are two real ISPs with two different towers with 5+ km distance, not VNOs (Virtual Network Operators) - but I see that somehow the connections share the bandwidth indeed. My guess is rather that either Balance 20x or FusionHub Solo has a hidden 55 MBit limitation. Although according to the spec, there is no limitation at FusionHub Solo anymore and there is a 100 MBit limitation on Balance 20x.
I wait with the upgrade to 8.1.0 RC1 until it is clarified and provide you with RA. Thanks a lot in advance!
Careful, that doesn’t mean that the backhaul bandwidth can not be shared.
Here in the UK we often see EE and Three, and O2 and Vodafone networks share physical infrastructure. We also have facilities management companies that get planning permission and install the physical towers and masts and the backhaul connectivity (microwave links and fiber generally) who then lease this to the mobile network operators. As such I will frequently see backhaul contention when combining EE and Three SIMs for example and both of these are full blown mobile network operators.
I have also done installs in the countryside here with a directional antenna on Three Mobile where I have connected in turn to different masts at different distances and different points on the compass only to prove that they are daisy chained together using microwave links and sharing the backhaul bandwidth. @Steve 's analysis here will be welcome.
The balance 20x has an advertised limit of 60Mbps VPN throughput encrypted. It is 100Mbps unencrypted. Have you disabled encryption at both ends in each profile?
There are ca. 10 towers in 5 km radius here. How typical is that the backhaul of multiple daisy chained towers of multiple providers is only 50 MBit in total? - which is enough only for a single subscriber. This sounds very exotic to me…
But I am open to any explanation which can be proven and lead to a solution…
I am also looking for an explanation why the UDP test of two parallel stream did not drive the connection with the specified speed. If I test with a single UDP stream, then it drives the connection exactly with the specified speed and this leads to packetloss. Something might throttle the speed in Balance 20X itself when two UDP streams are being tested. See picture again:
Cellular 2 (LTE12) is intentionally not aimed yet properly at the tower.
As next step I will upgrade to 8.1.0 RC1 to see if that Dynamic Weighted Bonding deals better with such quality difference between two LTEs.
To wrap-up here are the latest measurement results.
Changes in the setup:
put the external antennas to different sides of the building - one south-east and the other north-west - 5 meter distance was not enough on the same side to remove the cross-talk on B20 (800 MHz) band
Installed 8.1.0 RC2 and used “Weighted Dynamic Bonding”
Added external antenna to Cell 2 - reduced the latency big time
Replaced Cell 2 with a CAT6 modem - introduced carrier aggregation of primary band 3 (1800 MHz) and of secondary band 7 (2600 MHz)
I do not get why but there was a cross-talk on B20 (800 MHz) band when the antennas were on the same side of the building even with different ISP and with different tower
Weighted dynamic bonding increased somewhat the max bandwidth compared to Bonding
Any further proposal what I could improve?
What is strange: Dynamic Weighted Bonding seems to have better performance when both WAN connections are ON - BUT if only one connection is ON (for the measurement) then SpeedFusion does not seem to utilize the full available bandwidth of that WAN connection
@blade I downloaded RC2 and did some of the same experiments as you did. my conclusion for myself was…
Old Bonding works better than the new dynamic weighted (for me]
There is some secret relationship between FEC and the Bonding algorithm… it seemed to be a better able to detect when my crappy connection was full and back off with FEC set to low.
Latency difference cutoff had a significant effect. I went as low as 50ms and settled on 75ms for myself.
My connections
A. T-Mobile 40-50ms base latency 55-65Mbps middle of the night, As low as 1Mbps during congested times with latency spiking repeatedly to 400-450 when tower is deprioritizing me. Connection stable for 30days plus at a time.
B. AT&T 25-30ms base latency 130Mbps middle of the night. Always at least 95Mbps during congestion. A latency spike for it is 75ms. Disconnects 3 times a day on average.
All this said, I can push 100-120Mbps inside the tunnel and a little over 200Mbps outside (maximum)
Yes, my concern is also that bonding works as long as everything is good. BUT if one connection will have degradation, then the entire tunel might have degradation. Nevertheless I set the latency cutoffs now to 150 ms everywhere.
Regarding Dynamic Weighted Bonding. I see in the chart that it is somewhat better for me than plain Bonding. Not utilizing full bandwidth of a single WAN connection when only one of them might be a bug. Adding again the relevant chart:
Hi @blade, sorry for replying late, we are too focused to wrap up 8.1.0 for the next RC firmware. During the previous test cycle we have identified different issues related to the new Dynamic Weighted Bonding algo and we have fixed that now. 8.1.0 RC3 will be out soon, would you mind test it again when the firmware is ready?
Please let me know the result of the new algo, and to have more complete information, please use the “Export” button on the SpeedFusion Chart screen to save the PNG file, saving 2 different files from both local and remote devices will be better for me to compare the result, thanks!
Oh, and for the throughput test I’d suggest a longer duration (at least 30 seconds) to get a better understanding of how the network performs.
Hi @mystery, the new Traffic Distribution Policy - “Dynamic Weighted Bonding” is available for all devices with SpeedFusion Bonding capability. However for BR1 MK2 I remember it’s a Hot Failover device? In this case the new algo won’t apply, but one thing - the SpeedFusion Cloud, if you have SpeedFusion Cloud license we can activate bonding for the cloud connection, more details can be found here, we have a free trial tier as well:
MK2 does Speedfusion hot failover and Smoothing. Is there a difference between enabling Dynamic Weighted Bonding for Speedfusion Cloud and Fusionhub Solo? It would be awesome if you can enable it for Fusionhub solo cloud connection. Otherwise, can you please, let me know the reason you can’t?
@blade, please leave all the parameters (e.g. cut-off latency) as empty, the new algo DWB doesn’t work like the default bonding algorithm, many parameters have a different meaning, we don’t have a new set of params designed for DWB yet that’s why we are still hiding it in the support.cgi page. Usually cut-off latency cannot be set too high and using default empty value usually give the best result. I’d suggest to leave all the params empty and test again.
And also, is it possible to enable Remote Assistance (RA) so we can check directly on your device? In this way we can help to fine tune and see whether we can further optimize the algorithm. After enabling RA, you can send me the Serial Number via forum message, or create a support ticket then I’ll pick it up, don’t post it publicly on the forum. Thanks
