Outbound Policy Rule using Domain Name cuts bandwidth in half

See Balance One Core Underperforming Throughput for the backstory here

There seems to be a serious bug in Oubound Policy rules

  • If you are using firmware 8.0.1 or higher, and
  • you have an Outbound Policy rule
  • and that rule uses Destination as Domain Name
  • then your download speed is roughly cut in half.

Furthermore:

  • the slowdown is evident even if the rule is disabled - suggesting the CPU is wasting time evaluating rules when they are not even in use.

Things which don’t help:

  • the actual domain name used doesn’t seem to matter.
  • disabling the rule does not help (see above)
  • setting the Source address to Any or IP doens’t help.
  • having DNS caching on or off

Workaround:

  • Don’t use a domain name, instead use IP address as the Destination.

It would be great if this could get fixed in time for the 8.1 release

3 Likes

Has anyone at peplink confirmed this? Have you opened a support ticket regarding this?

I don’t see this behaviour on my Balance One (8.1.0 build 4929).

2 Likes
  • Jonathan: I have not opened a ticket on this yet

  • Martin: it’s 100% repeatable for me, I can add this one Outbound policy rule and the speed is cut in half, remove it and get full speed.

  • To see the bug, it looks like your WAN has to have > 300mbps speeds to show the slowdown.

  • What speed are you testing at?

Additional notes:

  • I have HTTPS Persistence on (set to By Destination, but I’ve tried By Source and it makes no difference)
  • My Default Outbound Rule is “Least Used”. My two WANs are very different speeds (one is 25mbps and the other is 480mbps) so “Least Used” ensures that the fast one is consistently used for speed tests.
1 Like

I noticed my bandwidth has been cut down in half the other day but was trying to figure out if it was due to me:
a) enabling Advanced > QoS > Application: for WiFi calling by single port 4500 UDP set as “high”
b) enabling bandwidth Advanced > QoS > Bandwidth Control
c) creating an outbound rule with a domain name, now disabled, but same speed issue
d) created a Speedfusion sub-tunnel in addition to my primary tunnel

I don’t think it was b) because I was running speed tests without issue after activating that. I think a), c), or d) introduced. Have to do some further testing but I’ve made a lot of changes this past week…

Can anyone say whether enabling any of the above A thru D should cut bandwidth/speeds down in half or even impact at all? I can understand some overhead but a 50% speed reduction seems high…

Thanks!

1 Like

In my testing, Disabling the rule doesn’t help - the presence of the rule (disabled or enabled) triggers the bug. Can you try deleting the rule and see if that fixes it?

When the bandwidth is cut in half, in my testing the CPU is not running at 100% (which is what I’d expect if the rules were causing too high of a CPU load). I feel like this is not just normal overahead, but some sort of bug.

1 Like

@soylentgreen I have created a ticket for you and please send us the information such as diagnostic report over there.

1 Like

@mystery

The more features you turn on , definitely it will impact the overall performance. For the 50% speed reduce, can you please open a ticket for support team to check on that as well ?

2 Likes

Thanks, that makes sense. I disabled QoS for my wifi calling and that seemed to help quite a bit. I will do some further testing when I have a chance.

Will do. Please note that there seem to be several issues (possibly related?). My issue is with Outbound Policy rules, but other people are reporting issues with QoS settings (which I haven’t seen myself). We may need more than one ticket.

Note that I’m seeing the slowdown even with the rule disabled which doesn’t make much sense, and suggests some other kind of bug.

@soylentgreen

Thank you for the reporting the potential issue that “Domain Name policy cuts bandwidth in half”.

Engineering team is investigating on this and i will update again when getting the latest info from Engineering team.

3 Likes

Confirmed this is a known issue for firmware 8.1.0. Detail info please refer to the firmware released notes.

1 Like

Thanks. Any idea when a fix will be available?

1 Like

When testing this please note that the behavior is weird:

  • the slowdown is evident even if the rule is disabled

In my testing, simply having the rule at all (disabled or enabled) is enough to trigger the slowdown .

1 Like

Having this issue as well… since upgrading to 8.1 to use the UDP port forwarding, my CPU load is constantly at 100% and my WAN speed is down drastically.

To give an example… I went from an ASUS RT-AC66U (minimal config, admittedly) and my speeds were 300Mbps (which is what I am paying for from ISP).

Then I swapped to my Pepwave SOHO and I am at 60Mbps max.

I do have a lot of firewall rules, event logging, and other things known to shew up CPU, but I turned all those off for testing and the CPU usage is still at 100%.

@soylentgreen

This is a known issue that mentioned (Listed in the released notes). For temporary workaround, please remove the domain outbound policy and reboot the device to clear all the state before you test again the performance.

@mamc

Surf SOHO MK3 only rated as 120Mbps router throughput, this can become a bottleneck for the network, would you consider Balance One instead ? Make sure you check the product feature before you consider the model.

1 Like

Part of the problem is most of my firewall rules are domain-based. I have many IoT devices and they all use AWS IP ranges. One of the devices connects to an AWS service which, if I could not use a domain-based policy, would have 100 different subnets (and those were ones I went through the trouble of summarizing myself). Rather than have extremely complex rules, I figured it would be more efficient to take advantage of the domain option. But yes in the mean time I suppose I could just permit all outbound traffic from my IoT vlan…

Regarding the Balance One… would I be able to restore a Soho config to it? Or at least is there some way to export firewall rules, dns records, dhcp reservations and grouped networks? That has been my biggest hesitation is reconfiguring a new device from scratch.

Thanks for your help!

1 Like

@mamc, Balance One and SOHO are different platform. You can’t restore the SOHO 's config into Balance One.

Since you mentioned you are having domain firewall rules. Can I confirm maximum 120Mbps (SOHO maximum throughput) is acceptable if changing hardware is not your option? We are preparing a special firmware to fix the known issue.

1 Like

Yes, 120Mpbs is acceptable! It would greatly benefit to get back and running as it was before. Thank you so much. I eagerly await the fix.

My B380 is running 8.1 with multiple outbound rules based on domain name. I don’t see any speed problems. Right now the CPU is at 7%. Apparently the issue does not affect all models.