MAX BR1 & IP Forwarding over bridge

So I have a setup where I have a Ubiquiti UDM Pro, a Ubiquiti Nanostation Loco M2 and a boat with a Ubiquiti Bullet M2 (AirOS 6.8) and a Peplink Max BR1 router. The Nanostation M2 is setup as a wireless bridge to the Bullet M2 (transparent bridge mode) to the WAN interface of the Max BR1 router. This works and clients on the boat can browse the Internet through the Bullet to the UDM Pro. My issue is no clients on the UDM Pro subnet can communicate back through the Bullet to the BR1 LAN side.

The BR1 is setup with IP Forwarding enabled and DHCP on the WAN interface where it gets an IP from the UDM Pro over the wireless bridge. From the BR1 LAN subnet [192.168.2.0/24], I can ping any IP on the 192.168.1.0 subnet as well as the internet. From the UDM Pro subnet [192.168.1.0/24] I can ping the wireless bridge (both sides) and even the BR1 LAN address [192.168.2.1]. I just cannot reach any IP on the BR1 LAN side [192.168.2.0/24] from the UDM Pro subnet. Here’s a diagram of the setup.

I checked the firewall rules on the BR1 which is set to any any any on the inbound. What am I missing here?

1616-bridge.jpg

Also want to point out that the UDM Pro has a static route defined to network 192.168.2.0/24 via gateway 192.168.1.253 [BR1 WAN] which is how I’m able to ping the BRI LAN IP [192.168.2.1], just nothing past that.

One other item that I’ve noticed is that the BR1 WAN gets a static DHCP address [192.168.1.253] from the UDM Pro over the wireless bridge. It also has a default GW of 192.168.1.1, which is how clients from 192.168.2.0 can browse the Internet. But, when I go to System and do a Traceroute from the WAN connection to 192.168.2.30 for example, it see that the first hop is 192.168.1.1 and the rest times out! It seems that the WAN connection doesn’t know how to get to 192.168.2.x.

Could this be the issue?

I assumed the UDMpro had the correct static route.

The problem is with modern networks and their design assumptions.

Modern Hosts only expect to have one router on a subnet. Only routers can be assumed to have a correct full routing table and to be able to speak to other routers and the networks beyond them.

Your traceroute implies the routing issue at hand here.

You will need to do packet captures to get the full feel of how the system is attempting to get the packets where they need to go, and I don’t have a UDMP to see exactly how it handles ICMP redirects. I know that a Juniper SRX would actually get the packets through but would generate 3 packets for every 1 sent, and the Fusionhub would send an additional packet.

Basically, hosts assume a network that only has one router. and routers should be connected directly to each other.

To show that this is the problem, add a static route for 192.168.2.0/24 via 192.168.1.253 on your test host. Ping will now work.

Now we don’t want to configure that on every host, and IPv4 did have a solution to a network with multiple routers, (google ICMP redirect) but modern operating systems ignore ICMP redirects for security purposes. Therefore we can’t have a network design that would generate them.

Therefore you should set up another network (192.168.3.0/24) that does nothing but connect the UDMP and the BR1, with the Ubnt wireless bridge systems.

Then all of the packets will go, Host → UDM → Bridge → BR1 → boat lan…and back

I first noticed this behavior with FusionHubs. but it really is enforced by the hosts. They only want and work with one router… you must give them a network that only contains the default router, no others.

I’m not 100% sure it’s the route on the UDMP side. Adding the static route on the PC did not work (that was the first thing I tried). Keep in mind, a PC on the UDMP side can actually reach the LAN interface of the BR1, but stops there. I actually added the FW rule to track this and sure enough, it sees the packet coming in from 192.168.1.16 to 192.168.2.30 which is allowed (below).

I’m not sure what happens when the traffic hits the WAN/LAN interface that’s destined for the 192.168.2.0/24 subnet. This is not even reachable from the BR1 WAN interface itself.

Then break out the packet captures. It saves time all around. PCAP on the requesting host, the UDMP and the Br1 and compare the packets.

the Asymetric packet routing is certainly generating extra packets. Looking at the MAC that you removed from the FW log would tell you if that packets came from the UDMP or the host.

Crisis averted! It seems that the client that I was trying to ping was NOT even online. I am able to ping the others. Thanks much!

br-1-client.jpg