Incontrol down again?

jo.campus · February 1, 2018, 10:29am

Hi,

Incontrol2 is taking a long time to load the devices, shows different reports. Is it down right now?

Thanks,
Josephine
Technical Support
SinglePoint

Tim_S · February 1, 2018, 11:23am

Working good for me…

jo.campus · February 1, 2018, 12:17pm

Hi,

This is Incontrol Mars. We experience this in different groups and organizations, device list is taking a long time to load, sometimes we receive error message and need to sign out and log back in, Clients also experience it on their end, device list shows 100 instead of 500. Please see the attached file. Error%20IC2

mldowling · February 1, 2018, 12:27pm

Hello @Tim_S & @jo.campus,
I can confirm normal operation with our organisations within InControl2 on both the Earth & Mars servers from here in Australia, everything appears to be loading normally and navigation is unaffected.
Happy to Help,
Marcus

jo.campus · February 1, 2018, 1:33pm

Hi,

I was also in the process of moving the devices from a group to their new organization. It takes a long time remove it from the group but when I try to add it to their own organization it says the device belongs to another org. I also encounter this error message from time to time.

TK_Liew · February 1, 2018, 6:47pm

Please PM the URL link for us to take closer look.

JamesPep · February 1, 2018, 8:40pm

While IC2 isn’t down down, we appear to be having some intermittent backend issues that are slowing things down significantly, that appear to have started a bit before @jo.campus posted the original comment.

We’re currently still investigating the root cause, and will try to get a fix tested and deployed ASAP

dbruce · February 7, 2018, 12:27pm

Hello,

Is there a postmortem for the issues experienced? I have several customers who are requesting additional information on the cause of the outage and preventative steps for the future.

JamesPep · February 8, 2018, 1:07am

It seems we ran into a capacity issue on our Mars organisation db caused by a spike in usage. As a hedge against this in future, we’ve shifted some of the traffic to read-only replicas and increased the capacity of the primary db instance.

jo.campus · February 14, 2018, 9:52am

Hi,

Incontrol2 is running slow again.

dbruce · February 14, 2018, 10:20am

Hello,

I am unable to load any organization on IC2. I receive “we are experiencing an issue” message.

jo.campus · February 14, 2018, 11:50am

Delete post.

mldowling · February 14, 2018, 5:40pm

Hello @jo.campus
InControl2 for both the EARTH & MARS servers are working OK for us from here in Australia.
Happy to Help,
Marcus

sitloongs · February 14, 2018, 8:17pm

Please refer to the forum post below:

jo.campus · February 27, 2018, 10:56am

Hi,

We are experiencing an issue with Incontrol2 again. Its taking a long time to load the devices. It seems that these slowdowns seem to be happening a lot lately.

jo.campus · February 27, 2018, 3:04pm

Hi,

We got the same error message again.

Michael · February 27, 2018, 5:26pm

The system was less responsive from 16:00 to 23:04 GMT+0. At 23:04, we restarted a component and the system recovered afterward. The captured error screen was returned during the restart. It lasted for less than 1 min.

The cause has been identified. We will prevent the same problem from happening. We are sorry for any inconvenience caused.