Balance design review


#1

I have a Peplink balance One Core and a Peplink balance 30. I currently have two microwave links. One is lower latency and higher download speeds then the other. I have static public IPs on both links.

When I play XBox, my wife ALWAYS wants to load Facebook while watching Netflix and doing some shopping on Amazon. I can separate the Xbox traffic from the Home traffic with an outbound policy; but every now and again - my games performance starts to jitter. Ping spikes occur.

Tonight, I setup my balance 30 (it was under my desk unplugged) and configured it so that I can unplug the Fast microwave link from the balance One and plug it into the balance 30. Then I move my Xbox wire to the balance 30. Now, no matter what my wife does, it won’t effect my game.

I don’t like having to physically unplug stuff, so I am asking for some design help. I would like both routers to be accessible to any device on the home network. I want both to be able to do DHCP (so I can leave the Xbox to Dynamic). I want to physically separate the traffic while playing the Xbox. I was thinking of plugging in the balance 30 to one of the WAN ports on the balance one and just “double hopping” traffic to that wan link. I would have the Xbox on lan1 and the fast wan link on WAN1 of the balance 30. Lan2 would go to the balance one. Then, during Xbox time, I can just “disconnect” the WAN1 port on the balance one to stop all traffic from going to it except for the Xbox.

Does anyone have any better solutions?


#2

I have 2 suggestions.

Using Balance One Core only

  1. Using Priority rule (Terminate Sessions enabled) with source IP
    1.1 Prioritized Xbox source IP to WAN1 then WAN2.
    1.2 Prioritized your wife’s laptop source IP to WAN2 then WAN1.

Using Balance One Core + Balance 30

Please find the setup below:

In normal condition:
Xbox —> Balance 30 (W1) —> Internet
Laptop —> Balance One Core (W1) —> Internet

If WAN1 of Balance 30 down:
Xbox —> Balance 30 (W2) —> Balance One Core (W1) —> Internet
Laptop —> Balance One Core (W1) —> Internet

If WAN1 of Balance One Core down:
Xbox —> Balance 30 (W1) —> Internet
Laptop —> Balance One Core (W2) —> Balance 30 (W1) —> Internet

Hope this help!


#3

@TK_Liew thank you so much for taking the time to do that diagram. That definitely is better than what I have come up with on my own. This week has been tough. Source code repository failed on me. Every HA solution has had some kind of problem. I haven’t much time for network stuff.

I really like how it is fault tolerant and fully redundant too. Hardware and link redundancy - gotta love it.

I have other outbound rules and networks, but I can just add a cloud instead of a device - simple enough.

:+1:


#4

I had some time today to tinker. I got it to work, but I think it can be better.

The way you suggested leaves me unable to ravers from one network to the other. I can get to the WAN links, but I can’t get to the management page of the balance 30 while I am on a network attached to the balance one. In the short term I set up identical VLans on both routers and created some static routes. This works, but I end up with 4 ports being used. 1 wan and 1 LAN on each router for the internet jumper, and then another LAN port on each router for the LAN jumper. Is there a better way that does not require the shared VLan and static routes? I tried adding a static route to the WAN IP, but the software wouldn’t let me.

Router untagged LAN IPs are 192.168.20.1 for the balance one and 192.168.19.1 for the balance 30. I am using IP forwarding on the WAN jumpers (no need to double NAT).

Would an OSPF area be a better way to implement this multi-router setup?


#5

I have also found what I believe to be a bug in Firmware 7.0.0.0 build 3310 for the balance 30. Well, there is a difference between the “Port Settings” page between the Balance One and the Balance 30.

On the Balance One, I am able to create a trunk with tagged VLans and the Untagged LAN. The Balance 30 will ONLY let me include the VLans in a trunk. This is causing a one way traffic issue going from router to router from devices on the untagged lan.

My problem is that I am on the Balance One in the untagged LAN and I want to get to the Balance 30 on it’s untagged LAN. I am pretty sure that my packets are making it to the Balance 30, but since it doesn’t allow untagged packets on this trunk - the return packets are being dropped.

It could be something that I am missing, but I am pretty sure that this is a bug. Can anyone explain why the untagged LAN can be included on a trunk in the Balance One but not on the Balance 30?


#6

Another status update. I was able to get rid of all VLans to get this implemented. It works well. Here is the layout.

Balance One Core (B1) - untagged 192.168.20.0/24 - 192.168.20.1 is the router LAN IP

Balance 30 (B30) - untagged 192.168.19.0/24 - 192.168.19.1 is the router LAN IP

Abbreviations
W1, W2 == Wan link number
L1, L2, etc == LAN link number

B1W1 (192.168.20.2) --> B30L4 (access any)
B1W2 (SlowLink public IP) --> Internet
B1L1 (access any) --> WAP1 (multi SSID and VLan)
B1L2 (access any) --> layer 3 switch (entertainment center) --> WAP2 (multi SSID and VLan)
B1L3 --> WAP3 (multi SSID and VLan)
B1L4 --> thermostat
B1L5 --> windows laptop running PRTG
B1L6 Empty
B1L7 Empty
B30W1 (FastLink public IP) --> Internet
B30W2 (192.168.19.2) --> B1L8 (access any)
B30L1 --> XBox1S

I am thinking of throwing a wireless access point on the balance 30 just so I can do troubleshooting without having to plug in a wire.

I used IP forwarding instead of NAT for the WAN jumpers.

One thing that is strange is the throughput graphs and client lists. When I do a Speedtest from my phone through to the fast link - I only see the traffic on the fast WAN socket. Even though it is going through the WAN jumper B1W1, I don’t see the traffic represented in the graph. I also don’t see my phone on the client list on the B30. I would think that it would have shown up since it was actively using a WAN link.


#7

jmjones:

Mind if I jump in here with a question? I’m wondering if you tried Mr. Liew’s first suggestion? I have a similar challenge – sending audio back and forth between HF radios hundreds of miles away. Latency and jitter are variables I really care about. Using a Balance 20 I have been successful in taking the very simple approach first suggested. In addition I set QOS to favor my applications.

There was one very limited situation where the Balance did not seem to do a great job and I responded by simply restricting the “offending” device to the higher latency WAN (“enforced” in “Outbound Policy”). [Comment: This is not intended to be a criticism of the Balance – overall, I’ve found their performance to be stellar, particularly given their price points.]

I’d appreciate your thoughts.

Rick


#8

I ran that way for a while, but when the slownlink gets saturated, I would see some lag spikes in game. There are many many more devices on my network. Many of which SPAM UDP garbage every 5 seconds. These multicast floods are impacting stuff, so segregating that link physically seems to work well - I just need to iron out these pesky details. My links are non-cir, so the saturation logic doesn’t work as expected for me. I tried with every device forced to the slow link and only the Xbox on the fast link (outbound policy enforced) and I would still see these spikes. I blame the large number of apple devices on my network mainly.

I was hoping the balance one would handle the spam better than the 30, and it does - it just works better physically segregated.

I am now thinking of a layer 2 pepvpn (unencrypted) tunnel between the two that I can disconnect while playing. I think I have to connect WAN to WAN to do that, but I think I can get by with just one cable. Right now outbound traffic goes out the WAN, but return packets seem to be coming back on the LAN. I can only get to both routers from either side if both connections are up (W1->L4 and W2->L8).


#9

OK. I very much appreciate your experience/comments.

Yes, Apple seem to be very “noisy” in broadcasting. And their “background chatter” is unbelievable. Side note: Of the 24 or so devices on one network I’m closely watching the worst – far and away – is the Davis Instruments Vantage Pro weather station. Horrible. I did not realize the extent of the problem until I built and implemented a “pihole” [https://pi-hole.net/] and saw the huge number of DNS inquiries. The pihole log told the story.

GL and thanks again for your thoughts.

Rick


#10

I have a TV that is sending out SSDP packets looking for a boot image server. It sends one out every 3 seconds. All day, every day. Unfortunately, many screen mirroring clients require a single multicast domain using SSDP (no VLans); so I can’t even block it by segmenting it. I could write a one direction firewall rule I suppose. I typically like to leave all internal traffic set to allow.

I think I am really over complicating this (dual Peplink setup). I am thinking of trying it with both WAN links running on the Balance 30, and then using the Balance One to manage the LAN and wireless LAN. What would be the most optimum (not fault tolerant or redundant - strictly performance based) way to do this?

Please don’t think me dogging on the Balance 1 or Balance 30. Both routers are totally capable of doing the task at hand without help from one another. I am just looking for any improvements in latency jitter in my Xbox. Remember that I can then plug the XBox in to the Balance 30 which would make it have priority to anything coming from the Balance1. I could even have the slow link attached to the Balance 1 in addition to the Balance 30 and use it as a backup link and only enable it when I toggle an outbound policy, disconnect a link (through the UI), or toggle a firewall rule. I have another couple routers laying around that I could throw in the mix to provide a constant (albeit only on failure detection) link to the internet if needed. Any and all help is appreciated. I have been looking into whether PepVPN may be of some use. I love having all the options. Thanks again


#11

@jmjones, do you think isolate the Xbox with other devices with Vlans will help? Please find the diagram below.


#12

I have tried several different configurations. I have 24 devices on my network and 90% of them using multicast spam for discovery.

Currently, I have it set up so that the balance one is managing the LAN and the balance 30 is managing the WAN. This has proven to be the best performing setup.

Unfortunately, I have to use some static routes to get from B30 to B1. This is because the B30 Wan link is connected to the LAN side of the B1.

With this setup, I am getting better throughput to both WAN links.

Could OSPF or RIPv2 be used to keep from using the static routes to the B30 VLans?


#13

I think I have finally come up with something that works like I want. There is only one drawback that I can find and I am hoping someone can chime in to help. Or to chime in and tell me what laws of the universe I am breaking.

Here is my new setup.

Balance30 WAN1 connects to fast link. Uses NAT.
BalanceOne WAN1 connects to slow link. Uses NAT.

BalanceOne is my AP controller. 95% of the network devices attach to this.
Balance30 has become Xbox router. I route traffic to it from the BalanceOne when I am not playing XBox.

B1W2 connects directly with B30W3. Both ends have a /32 address and use the peer as their gateway. They are both set for IP Forwarding. I should be able to just set up OSPF to exchange the routes - but I simply cannot get it to work. So, I configured a PepVPN tunnel - now the routes propagate and life is good. Everyone can get to everywhere (until I build out some control firewall rules). Like I said - life is good.

Buuuuuuut, now on the Balance30 there is only 1 client listed - the Xbox. Hmmmmm, that is strange - the BalanceOne is showing active sessions and the throughput graphs look correct. I believe that traffic is going that way. I look at the IP recorded on the speedtest - this data definitely went through the Balance30 and out the fast link. There is no obvious indication to let you know that traffic is going through this connection. The only indicator is the throughput numbers on the dashboard in the webUI.

And on top of that - my speeds - download and upload - damn near doubled. I am getting faster than what the ISP is selling me. What the deuce? I have two links - 16/4 and 26/4. I had an aggregated download speed of over 50 Mbps!

I have a feeling that the Balance30 was struggling to manage the WAN, LAN, and all the garbage flying across it. The BalanceOne did a bit better, but joining them together has made a tremendous improvement for me.

For those wondering, I called the ISP - WebFormix in Oregon - and they confirmed that all of the settings are appropriate. I am truly a happy camper.

Can anyone shed any light on what it would take to get the client list and active session stuff to show up in the B30?

If anyone has any theories on why this would be a better performing solution - I am all ears. I added a hop in the chain and added a unencrypted PepVPN tunnel and throughput increased and latency decreased.


#14

@jmjones

Seem this is a customize setup integrating 2 Balance router, would you able to turn on RA access for the Balance One & Balance 30 for us to further check the following:

  • Why speeds shown near doubled.
  • OSPF traffic route for the WANs (Balance One and Balance 30)

Regarding to your question below:
Can anyone shed any light on what it would take to get the client list and active session stuff to show up in the B30?

Answer:
This should be expected as clients list only show from LAN to WAN while your customize setup is routing the traffics from WAN to WAN.


#15

Thank you for the offer. The Balance30 is no longer in warranty, so I am just going to tinker some more to see what is up with the OSPF. I think I need to only set one device (Balance1) to be zone 0 and the Balance30 to be zone 1. I think. I will find out. For now, the unencrypted PepVPN is working like a charm. The bottlenecks are always going to be the internet connections anyway, and I only lost about 20% throughput. That is a guess based on throughput tests at 70Mbps.

As for the client list - that makes perfect sense. I would expect that the sessions list would have some indication of the connections though. Perhaps they should show up as transit connections?

Thanks again. I will draw up a network diagram soon and you can create the environment in your lab. It works great for me.