Hi @adnan.zafar,
How the latency algorithm works is explained here.
“Periodic latency checking packets are sent to the WAN connection. Latency will then be determined by the response time of the second and third hops. The traffic matching this rule will be routed through the healthy WAN connection with the lowest latency.”
So in your case depends on where the 'second and third hops" are located: in the Internet or not.
Regards,
Rolandas