Problem with FW 6.1.2 build 1597 on Balance One

Product Discussion Peplink Balance
1

I have two clients with issues running FW 6.1.2 build 1597 on Balance One routers. The keep having Internet outbound only dropouts. Inbound is fine, as I can remote in via Port Forward or VPN, just cannot get out to Internet.

Device Event Log shows “System: Time synchronization fail” when problem occurs.

I have tried replacing the TWC modem with no luck.

Alternatively, I have three other clients with Balance One routers running the previous FW 6.1.1 build 1269 without any issues.
Strangely enough, my personal router is a Balance 20 running FW 6.1.2 build 2717 without any issues as well.

So, it seems that the common denominator (from my experiences so far) is the latest FW release on the Balance One.

Is there a way to rollback to previous FW version until there is a fix?

FYI - I posted this same message on another thread in the Pepwave Max section but is was related to a different model router. So, I thought I would re-post here as well.

2

I noticed that there is a FW version of “fw-b1_hd4-6.1.2s24-build1598.bin” available on another post.

Is that a possible fix for my issue?

3

Hi Larussa,

The firmware 6.1.2s24 is meant to fix issue on environment that applying Outbound Policy with Domain name.

It would be good for us to look into your case closer by our technical support team, please log in the support ticket here.

Regards,
Wei-Ming

4

Already have submitted ticket.

5

FYI, we have been running two ONEs (hardware revision 1) on FW 6.1.2 b.1597 for about 10 days without any issues, that I am aware of, anyway. One is using cable modem service, the other is using a fiber service.

6

I was getting a “System: Time synchronization fail” repeatedly from two clients until a modem reboot usually fixes it but not always…running 6.1.2

Also, have three other Balance One routers in the field with 6.1.1 b1269 and no issues, so I concluded that 6.1.2 was causing this.

Perhaps it has something to do with geographical location. These are all residential clients in Los Angeles using TWC.

7

Update…

After rolling back firmware to 6.1.1 my dropout issues have gone away.

I hope engineering can find a cause for this so that the next FW release has a fix.

8

Hi Chris,

Can you turn on Remote Assistance to allow us to continue monitor you Balance One? If you can roll back to v6.1.2 will be great (if allow).

Thank you.

9

I just installed a Balance One running FW 6.1.2 build 1597. It worked fine overnight, but this morning it keeps going offline. When it goes offline:

  • all connected users lose both inbound and outbound internet
  • I can’t ping the device at all
  • the device will not respond to http: on the administrator interface IP

Now that this has started happening, I’ve power cycled the unit 3 times. Each time, it comes back up, works for about 5-10 minutes, then dies.

In the following ping log, I started pinging the device right after power cycling. After 25 seconds, it is finally booted up and starts responding to pings. However, after 174 seconds the device seems to crash, and it can no longer be pinged (this corresponds when we lose all inbound and outbound internet connectivity)

(Note: this device is set with an IP address of 10.0.32.1)

ping 10.0.32.1
PING 10.0.32.1 (10.0.32.1): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down
Request timeout for icmp_seq 0
ping: sendto: Host is down
Request timeout for icmp_seq 1
ping: sendto: Host is down
Request timeout for icmp_seq 2
ping: sendto: Host is down
…]
Request timeout for icmp_seq 13
Request timeout for icmp_seq 14
…]
Request timeout for icmp_seq 24
Request timeout for icmp_seq 25

at this point the device boots up and starts acting normally

64 bytes from 10.0.32.1: icmp_seq=26 ttl=64 time=0.643 ms
64 bytes from 10.0.32.1: icmp_seq=27 ttl=64 time=0.578 ms
64 bytes from 10.0.32.1: icmp_seq=28 ttl=64 time=0.544 ms
64 bytes from 10.0.32.1: icmp_seq=29 ttl=64 time=0.541 ms
64 bytes from 10.0.32.1: icmp_seq=30 ttl=64 time=0.554 ms
64 bytes from 10.0.32.1: icmp_seq=31 ttl=64 time=0.566 ms
…]
64 bytes from 10.0.32.1: icmp_seq=169 ttl=64 time=0.473 ms
64 bytes from 10.0.32.1: icmp_seq=170 ttl=64 time=0.488 ms
64 bytes from 10.0.32.1: icmp_seq=171 ttl=64 time=0.462 ms
64 bytes from 10.0.32.1: icmp_seq=172 ttl=64 time=0.517 ms
64 bytes from 10.0.32.1: icmp_seq=173 ttl=64 time=0.481 ms

at this point (174 seconds after boot up) it crashes, and we lose all IP connectivity

Request timeout for icmp_seq 174
Request timeout for icmp_seq 175
Request timeout for icmp_seq 176
Request timeout for icmp_seq 177
Request timeout for icmp_seq 178
Request timeout for icmp_seq 179
Request timeout for icmp_seq 180
Request timeout for icmp_seq 181

10

Something is going on here. Hope that engineering can find the cause.

FW rollback fixed my issue.

11

@soylentgreen Would you please create support ticket, and get a diagnostic report. So that we can send you the firmware.

12

Already done, see Ticket #744996

13

Dear all,

We understand this is a high priority issue. We are seriously looking into this. Your help in providing the diagnostic reports are much appreciated.

Thanks,
Keith

14

Thanks Keith - I’m happy to stay with 6.1.2 to help your testing efforts (as long as it’s working most of the time,) but since I don’t have the ability to roll back to 6.1.1, I’m a little nervous in case it crashes again over the weekend. Would it be possible for me to get a 6.1.1. download to have on hand “just in case”?

15

Hi,

Here you go.

16

Thank you. So far, i’ve been crash-free with 6.1.2 for about 24 hours.

When 6.1.2 crashed, I had been doing a bunch of configuration changes (mostly to the AP WiFi settings, since I was doing signal testing).

Since it’s stable (at the moment) I’ll stay with 6.1.2 - and only load 6.1.1 if I have some emergency.

Are there any tests I can run to help diagnose?

17

Hi,

We are requesting the Remote Assistance on the unit. Please follow up your case in ticket system.

Thank you.

18

Just to let you know, our two ONES have been running FW 6.1.2 b.1597 for nearly a month without any problems. I’m planning to install our third ONE next week.
I noticed that soylentgreen mentioned the Wi-Fi AP settings. If this helps with troubleshooting, I can tell you that we don’t use the ONE’s Wi-Fi AP capability, so we keep it set to OFF.

19

Update: my Balance One on FW 6.1.2 has now been running rock-solid for over a week, and I have made some small configuration changes during that time. When it crashed a week ago, it was after I was making a ton of changes including a number of changes to the WiFi / AP system.

So my guess is that the bug may have something to do with configuration changes, rather than just day-to-day operation? Does that sound right?

20

Update: I’m now at 21 days of uptime with 6.1.2.

So it really sounds to me like the problem is not one of operation, but shows up during configuration. Once the configuration is set, the Balance One on 6.1.2 has been very solid for me.