Need Active Queue Management for Bufferbloat (fq_codel)

Hi, we understood your concern and looking at it.

1 Like

@Mark9, yea we understand your pain and actually this topic has been discussed within the team for a long time, may be I could share more information with you before we have some firmware for you to play with. From our internal test result quite some time ago, fq_codel alone is just not enough for a dynamic environment, like a cellular connection. That’s why we are still working on it. So far we already have some candidate to help improve the problem, but not finalized enough yet to build into the firmware, we will absolutely keep you posted once we have more information to share with you. :+1:t2:

2 Likes

Very glad to hear this is in the works! As you point out, dynamically adjusting to a changing environment is advantageous. IQrouter is making waves with their router doing this. These days, it isn’t just DSL bandwidth performance that sags at different times of day. So does cable. Fixed ingress and egress values for fq_codel doesn’t work very well in that environment.

Note to others: I can state my experiences upgrading to DOCSIS 3.0 cable modems with 24x8 or 32x8 channels is that it seems to be better statistically for maintaining low latency than older cable modems with lower channel counts. Of course new DOCSIS 3.1 modems are now available now too.

Thanks very much for the update!

2 Likes

Does Peplink have plans to implement Active Queue Management in 2019?

Steve wrote “From our internal test result quite some time ago, fq_codel alone is just not enough for a dynamic environment, like a cellular connection.” Would Cake, the successor to fq_codel and which is now quite stable, take care of the issues you ran into? And is the version of Linux which Peplink is currently based on 4.4 or greater which apparently makes Cake installation straightforward?

If AQM isn’t scheduled for 2019, is there any possibility Peplink would consider adding fq_codel in 2019? It works well in many situations, raising dslreports.com/speedtest results to a grade of “A” for cable, fiber and DSL. As it is not unusual for routers to support multiple AQM choices, Peplink can add a more sophisticated AQM or two in the future. Fq_codel is built into Linux so it should only take adding configuration of fq_codel into the Peplink GUI.

The following (4 year old) article has a good summary of what is better about Cake than fq_codel: https://www.bufferbloat.net/projects/codel/wiki/Cake/.

The benefits include (taken from other articles):

“A major enhancement in Cake over fq_codel is replacement of the plain hash function with an 8-way set-associative version. Plain hashes are susceptible to the ‘birthday problem’ in which the probability of hash collision reaches 50% when the table occupancy reaches the square root of the table size (32 flows for 1024 queues), assuming a high-quality hash; we have also found that the hash function fq_codel relies on is suboptimal.”

The “Nat” option “Instructs Cake to perform a NAT lookup before applying flow-isolation rules, to determine the true addresses and port numbers of the packet, to improve fairness between hosts ‘inside’ the NAT.”

“One of the good things cake does … is that it reduces superpackets (GRO offloads mainly) to their packets which does wonders (vs fq_codel) to hold latencies stable.”

Fall back to dropping a packet if necessary for congestion control if ECN not working.

1 Like

Thanks for the info @Mark9. In firmware 8.0.0 beta 3, if you go to support.cgi (Login to Web Admin, in the URL bar rename index.cgi to support.cgi then press Enter), you’ll see there is a new “Mitigate bufferbloat” option like this, at the bottom of the page:

image

Please try it out and let us know your result! :slight_smile: You can get the new beta firmware here:

2 Likes

This is very exciting! I must be doing something wrong however as I don’t see “Mitigate bufferbloat” on the support.cgi page. I am on 8.0.0b03 build 3594 (Balance 20) and see “Lowest Latency outbound algorithm threshold [click to configure]” at the bottom of the page.

Please advise. Thanks

Sorry @Mark9, this is not available yet on Balance 20 since this is currently an experimental feature, we are still working on it. Do you have other models to test?

1 Like

I also have the Surf Soho MK3 if that will do the trick. I’ve been considering a Balance One, but don’t need the speed yet, so haven’t had a good reason to pull the trigger.

If the Surf Soho doesn’t do the trick, what models do?

Yea Soho MK3 should do the trick! Looking forward to your result. :slight_smile:

1 Like

Either I didn’t properly enable Mitigate Bufferbloat, or it didn’t make any improvements as measured by dslreports.com/speedtest. My DSL line has a D bufferbloat grade both before and after the mitigation. My Comcast line has a B or C grade before and after the mitigation. I can improve my grade to an A with a Ubiquiti EdgeRouter with their implementation of fq_codel (which they call Smart Queue).

The testing was with 8.0.0b03 build 1397 on my Surf Soho Mk3. I enabled Mitigate Bufferbloat. Afterwards I went to the Dashboard for Apply. I even tried rebooting to see if it made a difference, but it didn’t. Revisiting Mitigate Bufferbloat showed that the setting was “Enable”. I even set my WAN Upload and Download Bandwidth to values 10% less than the ISP provisioned speeds in case those values were used by the mitigation. Did I miss something?

As I will be traveling starting tomorrow until March, I will have limited ability to do more testing. I can remotely run dslreports.com/speedtest on my main Comcast line. I won’t be able to reboot as the Peplink router as there is some type of bug where my Comcast WAN spins with a Connecting message forever. My process for fixing this is to unplug my Netgear CM600 modem from the Peplink, power off and on the CM600, and then reconnect to the Peplink (yes, I need to open a case with Peplink sometime, but I was hoping some fixes for similar situations afflicting other Peplink customers would fix my situation). My fallback DSL line is rather slow and has high latency, something I would rather not subject others in our house to (or me when I remotely connect into the home network).

1 Like

I’m delighted to see peplink making strides to fix their bufferbloat issues for their customers! With a single checkbox to enable it, though, I worry. For the ISP uplink, the sqm (HTB+fq_codel) code and the later sch_cake require that the underlying framing be detected correctly in order to get nearly 100% of the bandwidth in all circumstances (dsl in particular has all sorts of framing issues, and we have a special docsis mode in cake). The upload/download speeds from the ISP need to be accurately measured before setting

once those three steps are done right, dslreports will tend to report A or A+ scores for bufferbloat, and advanced techniques such as ecn enablement (lossless congestion control) will “just work”. The flent tool (the rrul test is popular) can look into behaviors in more detail.

It’s unknown to me what wifi chipset (above about 40Mbit the bloat generally moves to the wifi) is in the “MK3 version of the Surf SOHO”. Google wifi’s evaluation of their version of the debloating wifi code is here: http://flent-newark.bufferbloat.net/~d/Airtime%20based%20queue%20limit%20for%20FQ_CoDel%20in%20wireless%20interface.pdf and has bloat vs rate/range plots at the end to die for.

I’d love to add this router to the bufferbloat.net supported list but more information is needed on how well whatever you’ve done works. Thx for paying attention, however, to addressing this problem, and I hope - if you’ve gone the extra mile to make configuration this easy - that one day you can just have it on by default!

4 Likes

Thanks @Mark9, is it possible for us to remotely check your device to get more details (Surf SOHO MK3 with 8.0b3 firmware)?

If possible, please enable Remote Assistance, and send us your Serial Number by creating a support ticket here:
https://contact.peplink.com/secure/create-support-ticket.html

1 Like

This is interesting @Dave_Taht. We for sure will look into it further and get back to you when we have some solid field test results come out. Really appreciate your hard works on this matter! :slight_smile:

1 Like

I just enabled the Mitigate Bufferbload feature using firmware 8 beta 3 on a Peplink Balance One. I ran a test using https://dslreports.net/speedtest and saw dramatic improvement in bufferbloat: from a “C” to an “A+” - roughly 200msec+ down to about 1msec.

Of note, the setting is a bit confusing: after you click the button to Enable it, you get a message that you have to Apply changes, but there’s no obvious button to click. Near the top of the support.cgi page is a link to the Dashboard, if you click that, you’ll get back to the main UI and can click Apply.

2 Likes

I’ve retried the same steps enabling Mitigate Bufferbloat that I did during my original testing with no change, steps which match what soylentgreen also did (successfully). My dslreports Bufferbloat score remains the same whether or not I am enabled or disabled (Surf Soho Mk3, Comcast, Netgear CM600 modem, testing through a LAN hardwired port). In this case, since I am running the test during the day, my grade was a B instead of a C or D at night. I observed that the Bufferbloat is occurring during the Upload phase of the dslreports speed test.

I went through several cycles of Mitigate Bufferbloat Enable and Disable, verifying the setting value was changing on the support.cgi page. I did not reboot and I am not in a position to do this as it will result in my Comcast connection hanging while I am traveling (as explained in a prior posting).

Remote Assistance is enabled. And I have created a ticket.

@Mark9

Thank you. Support team will followup with you via the created ticket

1 Like

Success! The support team found that I also needed to enable Advanced->QoS->Application->DSL/Cable Optimization. Once I adjusted my WAN download and upload values properly (around 10% below Comcast provisioned values), I was then able to get a Bufferbloat grade of A with dslreports.com/speedtest.

FYI, and as is expected with Bufferbloat algorithms, the extra cpu overhead reduces my download throughput on the Surf Soho to around 100Mbps +/- 5Mbps as measured on dslreports (the Surf Soho is normally rated for 120Mbps).

5 Likes

Good find - turns out I already had this setting enabled (on my Balance One, it’s under Network / QoS / Application / DSL/Cable Optimization)

2 Likes

Is this feature available on my Surf_SOHO device? I would really like to give it a try!

Thanks!

James…

@james_litchford

Yes, The feature is available for SOHO MK3.

2 Likes