Interesting. Can you open the live speedfusion profile graph in a new window and run the tests again then post a screenshot of the graph for review please.
Its the packet loss and jitter over the WISP connection that is screwing with your bandwidth when both WANs are active.
Because I’m lazy I would turn on forward error correction (in both directions) first and see if that helps.
Then if it wasn’t enough I would turn it off again and fine tune the suspension time after packet loss and cut-off latency values for the WISP connection which seems to be the trouble maker.
Start with a cut off latency of 130ms and a suspension time after packet loss of 100ms.
Once I have a stable 8.x device, I plan to do as aquablue is doing, on a boat too. Have LTE and WISP connections want to route over Speed fusion. Currently using fastest response on outbound policy. Subscribing to this thread to read aquablue’s experience and updated results!
Well, this is proving more difficult than expected.
For three weeks, a Verizon unlimited hostpot plan was performing very very well as a primary connection with speedfusion. I’ve been working on the suggested settings above during different times of day as the WISP connection fluctuated to try and get the verizon wan link and the WISP wan link to work together without the WISP dragging the verizon down.
A week ago I realized I was accidentally limiting the verizon upload speed because I had set the speed under wan settings to the maximum I tested with the day I setup the balance here, so I increased it from 10 to 25 and the verizon connection was then performing even better for a week on the upload side much of the day and the same during more prime times of the day.
But connecting the WISP still resulted in slightly less quick browsing than with verizon working alone, so have been running just speedfusion over the verizon link and plugging in wisp only to test.
My last step was to enable FEC, low on both the balance and fusion hub. I thought something was not working right, as it started getting worse that day. But I found something really odd happened with the verizon connection that day.
Now the verizon connection is getting a speedtest.net score of 29.6 mbps download speed, 14.09 mbps upload speed used without speedfusion, but over the speedfusion link I’m getting 1.10 mbps download speed, 12.34 mbps upload speed.
It’s as if speedfusion on the download side is being throttled only. I disabled FEC on both the balance and fusion hub, and then rebooted each for good measure even, but this condition remains and also seems to occur during both daytime and after 2:00 am (in case it was some kind of deprioritization affecting only the speedfusion port range). Very strange. I did the 8.0.0 to 8.0.1 firmware update during this timeframe too. Switching to an ATT sim the speedfusion speed is normal although ATT is performing a little slower here than verizon was previously. Not sure what is suddenly going on with speedfusion over the verizon link now.
In the “just a thought” category: Log into your Verizon account and check what they believe your usage to be.
Their “unlimited” comes with two limitations - deprioritization after (e.g.) 22 GB and a hard drop to 600 Kbps after (e.g.) 15 GB of hotspot use. (The thresholds depend on the plan you’re subscribing to.)
Could it be that they see your hotspot as having exceeded the latter limit?
Thanks. This is a prepaid unlimited hotspot plan that they sold from October of 2018 - May 2019 that I’ve kept “grandfathered” by continuing to renew at $70/month. I don’t think it’s ever shown a GB count in the verizon prepaid portal - it just shows U for unlimited GB on this plan. (it’s always deprioritized behind postpaid traffic.)
Speedfusion ran normally over it from May of 2019 in a test-only setup. I did probably pass 20 GB of usage this month.
It’s odd because if I bypass the speedfusion tunnel, download speed is still 30+ mbps over the verizon connection. But at the same time, speedfusion seems to suddenly now be limited in speed to ~1 mbps now, and only on the download side of the speedfusion tunnel while the upload side of the speedfusion tunnel is running at normal speed… very odd.
I did a quick search and didn’t find any other mentions of hitting a data use point where the speedfusion port would be throttled so much while speedtest.net would still measure normal speeds without speedfusion running (does speedtest.net measure port 80 or 443 or ?)
Ok, under the “Usage Details” on the side menu it lists each session with date, time, and MB used. I copied and pasted that into excel and then deleted all the cells with date and time leaving only MB used. Summed the MB used and came up with 50 GB.
I forgot that I got a new android phone and let it syncronize over this connection. That used about 30 GB of data two weeks ago!
And then I probably used 10 GB with speed testing to get it to work with the WISP!
If there is some threshold where it would throttle the speedfusion/vpn port, not sure if that will reset on a rolling 30 days or on my renewal date in a few days…
One day I was getting some packet loss on the verizon link; I had been getting packet loss on the WISP link since starting this thread. On that day testing with verizon + wisp I applied latency/packet loss cutoff values to the verizon link too. On that day it seemed to work great. Before, it seemed in testing that when I previously set a cutoff value on both WAN links it wouldn’t ever cut off the last remaining link, so I didn’t bother to remove it even though I’ve been having enough trouble with the WISP that I’ve mostly been running with the verizon WAN only.
Then as more people started coming to this area, the tower is getting a bit more loaded and it must have started tripping. Don’t know if upgrading the firmware had any role (it was fine immediately after, at night, but started running slow sometime after) It must have started tripping the cutoff, as removing latency and packet loss cutoffs for the WAN1 (verizon) has verizon now running great on the download side (speedfusion vs. without speedfusion) again at 34 mbps at 2am.
Its the packet loss and jitter over the WISP connection that is screwing with your bandwidth when both WANs are active.
Because I’m lazy I would turn on forward error correction (in both directions) first and see if that helps.
OK, back on track. I had planned to post some nice graphs, but took my time by creating a mystery for myself by winging it instead Learning slowly though.
Enabling FEC, low on the upload helped a lot, and is better under these conditions, at least when testing today, than the latency/packet loss on wisp.
With FEC disabled on the upload (balance) side, at 2:30 am:
Verzion Alone: 24.4 mbps upload
Verizon + WISP: 11 mbps upload
With FEC enabled:
Verizon + WISP 27.1 mbps upload
With FEC disabled and cut off latency of 130ms and a suspension time after packet loss of 100ms enabled on wisp WAN2 only:
Verizon + WISP 15.9 mbps upload
However, on the download side, FEC didn’t seem to improve things, but not much worse tonight either; however I was not able to get a download speed greater with Verizon + WISP than with Verizon alone and WAN2 disconnected tonight.
For now I have FEC enabled on both the upload and download sides.
traffic distribution policy overflow + FEC
download: 26.6
upload: 29.8
(with verizion as priority 1 and wisp as priority 1 overflow precedence 2)
Oddly overflow doing a speedtest downloads only over verizon but uploads over both connections. I’m not sure what would trigger the download side to “overflow” to wan2 that the speedtest doesn’t trigger.
High jitter is an affliction of many WISP systems, particularly those that are oversubscribed or have insufficient back-haul capability. So, not surprised. Sad but true.
Here is what it looks like (when both connections are individually at their best in terms of speed, in the early am hours.) This is with FEC enabled in both directions set to low, and no latency cutoff or packet loss cutoff set on either connection.
In speed test from two different carriers, you should probably CHOOSE the speedtest server and make sure that it’s not Verizon or WISP server. The speedtest app (server) may be causing your strange results. I’ve experienced this with Verizon and AT&T when one or the other had significantly higher/lower latency. Worth a try…
Yes, I also try a few speedtest.net servers in the same city as the FusionHub to do a triple check to make sure one speedtest site isn’t giving an odd result. Sometimes that is the case, but not with this.